The Art of Lean Software Development

This is an admittedly short book at only 122 pages. The authors felt that there was a need to have an introductory offering into the world of Lean and Agile methodologies, and have done a great job of keeping the book very focused. They offer an introductory look at some foundational Lean and Agile concepts and provide a clear understanding of how Lean is not in itself a specific process or methodology, but is at it’s core a philosophy that draws on existing processes and methodologies, giving them true value to the business.

The Good

The core of this book discusses what the authors believe are the “five most important practices that you can adopt as you begin your Lean software development journey.” The practices that they outline are common practices that are found in many existing Agile methodologies, so they are likely to be familiar to those who have done any research or read any blogs on recent trends in software development.

They actually list six practices, but consider the first one to be ‘Practice 0’ – what you should be doing, period, even if you are not doing any form of Agile or Lean development. The full list of development practices that they list, are:

• Practice 0: Source Code Management And Scripted Builds
• Practice 1: Automated Testing
• Practice 2: Continuous Integration
• Practice 3: Less Code
• Practice 4: Short Iterations
• Practice 5: Customer Participation

Over-all, I’m happy with the practices that they outline. The practices they chose are very fundamental to most Agile methodologies, and they do a great job of explaining how each of the practices supports the lean philosophies that were discussed at the beginning of the book. Each of the practices is discussed in enough detail from the process standpoint, to make the reader want to know more. They avoid getting into too many technology and platform specific details. However, they often mention frameworks and tools that cover a wide range of platforms, giving the reader ample information to continue their research. The authors also manage to be very up to date in the processes that they discuss, and often mention newer, budding efforts such as Behavior Driven Development and applying the Theory Of Constraints to software development.

While the practices may be engineering focused, they present them as benefits to the business of software development, which I applaud greatly. Far too many Agile books and discussions are focused entirely inward toward the engineers and project managers, with little to no regard for the customer (other than the standard ‘co-location’ or ‘constant communication’ lines).

Throughout the book, including the final chapter on ‘What Next’, there are multiple references to existing literature on the various tools, technologies, philosophies and practices, including Lean resources. The last portion of the book is dedicated to listing these references in a well organized, categorical manner. This makes it very easy for the reader to find additional resources on the information provided in the book.

Personally, my favorite chapter of the book was chapter 1: ‘Why Lean’. This chapter introduces many of the problems that we have known about in software engineering. They go on to talk about the Agile Manifesto and mention many of the popular Agile methodologies. The real meat of the chapter, to me, is the explanations used to justify Agile and Lean to the business – actual studies and reports generated from real world projects reported under ‘The CHAOS Study’, with over 40,000 (yes, that’s forty-thousand) projects in the study. I find myself referencing this chapter and this data time and time again. It’s a very enlightening dataset.

The Bad

There are a few questionable items in the book, in my opinion. I would not say these items are show-stoppers to prevent people from buying the book, though.

Code Coverage

The first item that jumped out at me is in Practice 1: Automated Testing. At the bottom of Page 49, they say the following (emphasis is mine, to illustrate where I take issue):

It is unrealistic, and probably not worth the cost, to achieve 100% test coverage of your source code. For new code and projects that are new to automated testing, 60 to 80% code coverage for your tests is a reasonable target, although seasoned testers will strive for 80 to 95% coverage. There are tools that will examine your source code and tests, and then tell you what your test coverage is (for example, Emma and Cobertura for Java, and NCover for .NET).

I am disappointed that they would make such a blanket statement without any conditions or explanation. I think the notion that 100% code coverage is ‘unrealistic’ betrays the authors’ personal experience and technologies that they develop against. Yes, there are many cases where 100% code coverage is not reasonable (integration with Sharepoint, for example), but there are plenty of cases where 100% coverage is possible and should be expected. At the very minimum, I would prefer them to say that starting with 100% coverage should be the default, and then to discuss scenarios where this is not reasonable.

Continuous Integration

The second item that jumped out at me was the overall repetition provided in Practice 2: Continuous Integration. While the chapter itself is very valuable and provides a great amount of information, it is very repetitious. They seem to take the following approach throughout the chapter:

1. List the benefits
2. Describe the benefits in a little detail
3. Re-list the benefits and how they support Lean
4. Re-describe the benefits in a little more detail and repeat from #3

It got a little old hearing the same thing over and over and over with just a little more detail each time. That being said, the chapter is still worth reading and properly understanding. They do a great job of describing why CI is important – but do it very repetitively (much like my description, here. )

Summary and Recommendation

From the preface, page ix:

Who Should Read This Book?

This book is for software developers and managers of software developers who are new to Lean software development and, possibly, new to Agile software development. It is for those who want to quickly understand why Lean software development is important and what it can do for you.

This is purposefully a short book, with short chapters. We know that you are just as busy as we are, and we don't believe in padding our chapters with useless fluff. In every chapter we try to be as succinct and to-the-point as possible. Our goal is to introduce you to the important topics and resources so that you know where to go when you need more details.

The overall focus on the business value that Lean provides, while introducing the engineering practices makes for a great project management or senior software engineer / team lead level read. There is enough information in this book to hopefully garner the additional research of the readers. I believe the authors have appropriately stated their audience in the preface and have done a phenomenal job of keeping the book short, easy to read, and very informative.

Final Score

In light of the two issues that I listed above, I would give this book a 4 out of 5 and recommend it to the same audience that the authors recommend.

In the manufacturing world, you would never find a company that assembles a bunch of parts into a final product before inspecting any of the individual parts, and they would not wait until the end of the assembly line to test for the quality of the product. The very notion of waiting until the product is “done”, to test it, would be appalling. How much time and money would be lost trying to figure out why something didn’t fit together properly, why it didn’t work, and why there was a quality issue with the final product? Rather, we see the manufacturing world taking an active role in preventing defects. Yes, they still do a final quality inspection, but the primary means of ensuring a quality product is delivered is not by waiting until the product is assembled to test it. They build quality in from the start and maintain that quality throughout the manufacturing process.

Prying Open The Case

Imagine the inner workings of the phone to the left. This is a very complex piece of technology. Do you think Cisco would wait until they have assembled this phone and then try to pry open the case so that they can insert a set of electrodes to test and see that the circuit board is connected correctly? I certainly hope they don’t. Instead, when a manufacturing company is building something – anything – they start with the idea of preventing defects, not waiting until they are identified and correcting them. Many companies have an active Zero Defects policy where defect prevention is paramount and quality inspection is almost just a verification of what they already know – that the product is defect free.

When a part is stamped out, formed, molded, or otherwise created, it is done so to an exacting specification. After the part has been created, the part is then tested against the same specification to which it was originally built. If the part does not fall within the tolerance and guidance of the specification, it is scrapped and a new one is made. If a series of parts are found to be out of specification, it’s usually a sign that something in the process, tooling, or other portion of the manufacturing process is not right. When this happens, they fix the cause of the problem – whether the machines need to be calibrated, the people running the machines need better instructions or whatever the cause is. In the end, the specifications for the parts were used to create the part, identify whether or not the part was up to standards, and decide whether or not to keep that part.

What’s more, the manufacturing company doesn’t wait until after they start creating parts to create the specification. Rather, they take the time to properly engineer the specifications up front. They take measurements, create prototypes with varying specifications to see what works best, record the success and failure rates of the various specifications that are tried, and use other design and engineering principles to scientifically calculate the exacting specifications that will be used to produce the parts. This occurs at all levels of the product’s design and creation. For every resistor, capacitor and microchip that builds a circuit board, each one of them has their own specifications that have been carefully engineered. If any single capacitor does not meet the specifications, it is not sent to Cisco with the hopes that it works anyways. Only when all of the specifications of each part are met will they solder the parts to the circuit board, creating a subassembly.

When a subassembly is created, it also has a specification to which it was built. The subassembly then undergoes the same quality assurance process – verification that it meets the specifications and operational requirements. The process continues from here – each subassembly gets connected to a larger system which is built to a set of specifications, with rigorous testing of the larger system as it is built, ensuring it meets the specifications. When the final phone is assembled, we don’t have to worry about whether or not a specific capacitor is soldered to the correct location – we don’t have pry open the case on this phone and insert a set of electrodes to see if the electrical current is flowing correctly. Instead, we only need to plug this phone into the correct connections (an Cisco IP phone system in this case) and verify that the phone actually performs all of it’s functions, according the functional specifications of the phone. There simply is no need to verify the capacitor that was used in the very first step. We know it works because it was built in a system that actively prevents defects.

The manufacturing world is obsessed with testing. They are willing to test from the lowest possible levels of the system, out to the end-product and the behavior that is expected. They do this because the consumers of manufactured products demand perfect. Why, then, are so many software development companies so willing to only test from one end of the process? To only test once, and only from the user interface, just before the product is shipped?

A Specification By Any Other Name

Unfortunately, the software development industry as a whole, is years behind the manufacturing industry. Our definition of quality and success are often skewed and we may consider fifty or more known bugs in a system of moderate to large size to be acceptable. It doesn’t have to be this way, though. We have the technical capabilities of following in the footsteps of the manufacturing industry, and we should.

I’m sure there would be no small number of people that would say we already employ the use of specifications in software development. After all, that’s what the requirements gathering phase is for, right? So many software development companies have put so much effort, time and money into the process of producing a piece of paper that can be understood by humans, and labeled this a specification. The problem we face with paper, though, is how to effectively verify the software against what the paper says. How do we verify that series of software lines and I/O statements that are understood by a computer have actually implemented the human readable text on the paper?

We are fortunate, actually. We don’t have to accept a Word document or a piece of paper as the specification to build to. We have the ability to create executable specifications! We can, and should, be creating specifications that can measure and verify our code. Most people call it test driven development (TDD). Some call it Behavior Driven Development (BDD). I like to think of it as Specification Driven Development (SDD? Not sure if that really exists. And really, do we need another xDD acronym?). We write code that exercises our code in the form of unit tests, integration tests, functional tests, acceptance tests, or whatever you want to call them.

We’re Not Just Stamping Out Parts

One of the major problems that I have with the manufacturing/software development analogy is the obvious statement that we don’t stamp out the same parts over and over again. In spite of my previous comments on this analogy, I now think that we are more analogous to a specific part of manufacturing than I had previously understood. A more accurate representation of software development in the manufacturing world is new product design and development. The parallels work quite well from this perspective. I am not going to expound on this in great detail at this point. It should suffice to say, for the moment, that the process of product development described in Wikipedia is a fairly accurate representation of what we go through for the average software development project.

When a manufacturing company is working on a new product, they once again don’t stamp out parts without knowing what they are doing. Many different parts may need to be tested, many different designs may need to be tried, but every one of these is still built to a specification. The major difference is that the specifications used are expected to change over time, until the final specification for the final pieces are accurate enough to produce a production-ready prototype.  The same notions can be applied to the software development processes in TDD, with some additional benefits.

Built To Specs, Regression Tests And Change

Change happens. It’s a simple fact of software development. A customer thought they wanted X, but in reality they needed X-1/B – not quite what we originally thought. When this happens, we once again have a significant benefit created by our executable specifications. We only need to identify those specifications that are now wrong, correct them, and change the affected portions of the system to match the new specifications.

Our ability to change is direct evidence to one of the many benefits of TDD: regression tests. Every specification that we write becomes a regression test the moment we fulfill that specification’s requirements. With this in mind, we can work with an almost reckless abandon, free to add features, remove features, fix bugs (because let’s face it – we’re still going to find some issues somewhere in the system) and refactor the system to a higher standard, all without worry of breaking the system. We can act with this level of confidence because we have a safety net in our regression tests. If (when) we do break something in our new efforts, we will be notified the moment we re-execute our specifications – that is, run our regression tests. A specification test will fail and we will have a clear indication of what failed and why. This deep insight into the system gives us even further confidence in correcting any issues that we introduce. When a failed specification test tells you exactly which value from which class is wrong, and the context in which that class was executed is known (the exact input and expected output), pinpointing the problem becomes a rote process. Fixing the issue becomes relatively simple, and we begin to see true productivity improvements in our processes.

Start With Quality, End With Quality

Our industry is currently suffering from a lack of quality. We ship horrendously bad user experiences in products that are late and well over budget, yet we call this a ‘success’. It doesn’t have to be this way. If we change our perspective and start to take some cues from the manufacturing and product design and development world, we can dramatically increase our effectiveness as software developers. We can create high quality, low cost solutions like the world expects from manufacturers. Built to specification is certainly not a silver bullet. It is, however, the definition of quality in a Zero Defect environment.

By employing a built to specification mindset in our software development efforts, we can start with quality and maintain that quality throughout the life of our projects. This is the same process that is undertaken when a manufacturing company is working on new product design and development. It works well, it’s a proven process, and most of all – it makes sense. Build your software to specifications. Just make sure they are executable specifications.

For so long there have been so many advocating the benefits of the various Agile, Lean, Iterative, or whatever-you-want-to-call-it-these-days methodologies. We, as software developers, seem to understand the benefits of these methods. So why, then, do we still have customers that insist on big design up front, waterfall based methodologies, and other practices that we know are wrong?

Some Ad-hoc Consulting

During my Christmas vacation, I had the opportunity to do some consulting / coaching for my dad’s current company. The employee list that I spoke to consisted of my dad, my aunt, and my cousin. It was a nice little family gathering, I got paid with some amazingly good barbeque, and it was a very enjoyable experience.

I’m a software developer working for a company that provides software development and related services. My dad’s company is a customer of a software development organization (not the one I work for), receiving some custom built software for a project. They have been running into some issues with quality and timeliness of delivery for the last year or so and were wanting to ask some very pointed questions about the state of the software development industry. Essentially, they wanted me to tell them whether or not the issues they were having are “normal” and / or “acceptable” when receiving custom software development services. My answer was an unfortunate “yes, this is normal”, but “no, this is not acceptable”.

Over the next few hours of our discussion, we touched on many different issues that they are having and how some of those issues can be addressed from their perspective and from the perspective of the software development supplier. In the end, I had successfully educated my dad’s company on a few key points, confirming their suspicions and observations, and giving some new directions to start heading with the company that is providing the development services. I can’t go into all of the detail, but the summary bullet points included the following:

• The customer won’t know what they want until they see something they don’t want
• The software provider should deliver a working, stable version of the system on a regular basis, allowing the customer to use it in it’s current form
• The customer sets the priority of features and functionality to be implementing, in coordination with the technical needs and pre-requisites of the software development organization
• The customer is allowed to say something is wrong, even when they receive what they asked for
• The relationship between the customer and service provider must be a whole-team view. That is, “we are one team. we succeed together and we fail together”

On top of these bullet points, I laid out a simplified Scrum process for them, based on the already stated 3 week development cycle that the provider prefers.

In the end, my dad’s company was shocked, astounded, amazed, and energized by everything they heard. The light bulbs were clicking in their eyes, and I could see the value that they received form the conversations. They have dramatically changed their understanding of what it means to be a good customer and what it means to be a good supplier.

A New Perspective

As much as I believe I helped my dad’s company in their understanding of what they should expect and how they should approach their issues and needs, I also learned a great deal from this experience. It was very enlightening for me to hear the issues that we have known for so long as software providers, but from the customer’s perspective. Nothing that they spoke of was unknown or unfamiliar to me. I have lived through all of the same pains and issues that they are going through – from both a software provider and a customer of software services.

What really stuck with me from this experience, though, is the idea that our perspective on how to improve our industry is likely wrong. Yes, we need to continue educating our own ranks on the benefits of better software development techniques and processes. However, it does no good for us to have the perfect pneumatic toolset when the customer is asking us to use a rusted hammer and a broken hacksaw.

What I’m getting at is that its not us, the software providers, that need to change so greatly. Rather, our customer base needs to be educated and change. We need to stop being so self-centered and introspective for a while so that we can educate our customer base on what it means to be a good customer; how they can accomplish their goal of receiving quality software that provides true business value for them, at an affordable price. Until we have a customer base that expects software to be high quality, the first time, every time; expects to be given working software on a regular basis; and expects to be able to recognize that what they thought they wanted is not what they really needed; we will have no chance of our inward looking self-improvements taking hold.

I have to say, the Agile Manifesto got it right from a values perspective. The values espoused there are what we need to be looking toward, but I think our perspective needs to be changed on how we enable those values to be reached.

Of course, if we do succeed in educating our customers, we need to be able to respond appropriately. So please, continue the education of our own industry. Create better development standards that increase our quality and productivity. If our customer expects a quality delivery every 2 weeks, we better be able to deliver a quality product every 2 weeks. Just don’t stop here. Educate your customers. Help them understand the financial impact of poor development standards and processes. Change the norms and improve our industry by creating a customer base that requires perfect. The manufacturing industry did it 20 to 30 years ago. It’s time for us to step up to that level.

Where Do We Go From Here?

So where do we go from here? How do we start educating our customers? How do we change the social norms of software development so that our customers expect us to behave more like professional craftsmen and engineers, delivering quality faster?

I don’t claim to have the answers and I don’t claim to be the first to encourage this change. I only hope that we as an industry will begin to understand that the customer is usually the one that sets the expectation for how often software is delivered, and that we need the customer to expect something better.

Imagine that there are three group of people going out for a hike. Within each group of people, we have a goal of everyone reaching a picnic table at the end of the hike (and no one can eat until everyone has arrived). One of the people in each group has a map and a compass, knows how to use them, and will be directing the group on where they need to be going (we’ll call that person “the Pathfinder”). Everyone else is just along for the hike and the picnic.

The Pathfinder in each group is given a specific set of instructions on how they are going to direct everyone else.

Group 1: Observe, Plan, Walk

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course in one hundred foot increments, with your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking. Give an updated direction to walk every one hundred feet, enduring the group can go around any obstacles such as large rocks, impassable tree groupings, and small creeks.
4. Once you have plotted the correct course, store the compass and map in your backpack with the understanding that you are not allowed to use them again, until you have reached the picnic area.
5. Have the entire group read the instructions, then set out on your hike.
6. You are only allowed to check that everyone is on course by re-reading the instructions that you wrote down, every 100 feet – when the team needs to change direction to avoid an obstacle.
7. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 2: Observe, Plan, Walk, Start Over, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet, ensuring everyone in the group understands the directions.
4. Place your compass and map on the rock, then hop down from the rock. Re-join your group and lead them in the right direction for one hundred feet
5. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
6. Travel the one hundred foot distance with your team, ensuring that they end up in the correct location.
7. At the end of the one hundred foot walk, have the group stop where they are.
8. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
9. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
10. With the help of the ropes to guide you, walk back to the rock at the beginning of the trails, by yourself, and climb back on to it.
11. Plot the next one hundred foot leg of the journey from where the group currently is, using your map and compass.
12. Repeat this process from step 3 through 11, until your entire group has reached the picnic area.
13. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 3: Observe, Plan, Walk, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area and general understanding of how to get where you are going, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet.
4. Place your compass and map in your backpack, then join your group.
5. Let the group know where they are heading (the end goal) and where they need to be in the next one hundred feet, ensuring everyone in the group understands the directions that you outlined.
6. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
7. Lead the group in the right direction for one hundred feet, ensuring that everyone is able to safely navigate any obstacles.
8. Verify that that your group has stopped in the correct location, by using your map and compass and any other landmarks or means needed.
9. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
10. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
11. Repeat this process from step 3 through 10, until your entire group has reached the picnic area.
12. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Predictions on Outcome?

• Which group do you think will make it to the picnic site first?
• Which group do you think will be the most happy with the way the group was organized and run?
• Which group of people do you think would be most likely to do the same process on another hike?

Observations of the Groups

Group 1 didn’t get to the picnic site very fast and wasn’t very happy about the experience. While everyone in the group was standing around waiting, the Pathfinder was sitting on top of a rock, writing things down in a notebook, trying to describe – from a very high level, where people were supposed to be walking. When the Pathfinder finally came down from the rock – possibly an hour or more after he climbed up there – the initial directions were good and people could start finding their way. As time went on, though, the detail of where they should be going became more and more unclear, because the Pathfinder could not nee the actual pathways below the tree tops and behind the large rocks. This caused Group 1 to constantly stumble and struggle, creating a situation where the team had to find paths around obstacles that they couldn’t previously see. Every now and then, they got lost because the description of the landmark was insufficient from the distance that it was recorded. They eventually got to the picnic site – well after everyone was hungry and tired. In the end, no one was happy with the experience and just wanted to go home.

Group 2 had an easier time finding their way. Since the Pathfinder and the group were able to see all of the obstacles within one hundred feet, the directions that the Pathfinder described was much more enjoyable. No one stumbled over anything or got lost. But it was a difficult journey, at times. The group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. The problem, though, is that the group took forever to get to the picnic area. At first, having the Pathfinder re-check the direction they were heading was great. Everyone knew that he was keeping the group on target. However, every time the group reached their next stopping point, the Pathfinder would gather information about the next hundred feet, across a wide arc since he didn’t know which direction to go next, with great certainty. Only then could he travel all the way back to the original rock to verify that everyone was on the right path, heading toward the picnic table. After a few iterations of hiking, the process of verifying the course and location took more time than the process of hiking. The Pathfinder was simply repeating the same hike over and over again, to verify that they were on the original path laid out. No one in this group was disappointed in the path that they took to get to the picnic table, but no one appreciated the standing around and waiting for hours on end – waiting for the Pathfinder to get the next one hundred feet plotted. When they finally reached the picnic table, group 2 was full of hungry, grumpy people who just wanted to go home.

Group 3 got to the picnic site in record time. It was a difficult journey, at times – the group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. With the ability to see all of the obstacles in the surrounding area, and the ability for the group to make decisions that would get them around obstacles while still making progress toward the picnic table, the group was successful in getting to the picnic table before lunchtime. Everyone on the group loved the trip and appreciated the ability have input in how they got to the picnic table. As long as the direction that they headed for the next one hundred feet did not cause a serious delay in reaching the picnic table, or was justifiable by avoid an obstacle, all input was considered before setting the new direction. Several of the people in group 3 decided that the would take a few more hours and head off into the next set of hiking trails, trying to reach another picnic table before dinner time. Other decided to bring their family out to the trail the next day, and re-walk the well defined path (noted by the ropes tied to the starting and ending points of each leg of the journey). Overall, everyone in group 3 enjoyed the experience.

The Real World

Now imagine that the Pathfinder is represented by some aspect of your organization. It may be the company or project leadership; it may be the customer; it may be an individual person or group of people within a project team. Your Pathfinder may not be a person or other sentient being, though. Perhaps its the Methodology (capital M, as defined in Peopleware) or political environment in your company.

Which Pathfinder(s) do you have in your company? Which Pathfinder would you rather work with? More importantly – if you are not working for / with the Pathfinder that you want, what steps are you taking to correct that? Be a catalyst for change – appropriately and professionally.

Continuously improve – always within the context of the goal.

In part 2 of my Kanban in Software Development series, I talked about completing a kanban board with queues, order points and limits. We saw how to take a complete development pipeline and work with a team, its processes and its bottlenecks. In the end, we had a kanban board that could easily represent the processes of the fictional team that we outlined.

One of the questions that I've often asked about a kanban board is how anyone would know when work in one column is done and ready to be pulled into the next column. For example - if a kanban card is sitting in the Analysis column, how does a developer know when that card is done so that they can pull it into Development and start coding it? I found the answer to this question when I was at the Kaizen Conference in October. Jef Newsom did a workshop on kanban and we ended up with this same question, and a solution.

A Pipeline for Analysis - WIP and Done

To facilitate the visualization of the difference between work in progress and work that is ready to be pulled to the next column, we can use the concept of a pipeline and split our existing queues into a WIP and Done step. For example, we want developers to pull work from the Analysis queue into the Development pipeline. We can show which cards are ready to move by splitting Analysis into sub-columns of WIP and Done.

When an Analyst is ready for work, they would pull from the Backlog into the Analysis / WIP column. When the analysis work is done and the card is ready to go into development, the card would be moved into the Analysis / Done column. Since we are wanting to maintain the concept of a queue for the overall Analysis column, we have create the WIP and Done subdivisions as a pipeline (noted by the dashed line). This allows us to keep our order point (3) and limit (5) in place for Analysis, and know what work is ready to be pulled into Development.

Applying Pipeline per Queue Across The Board

Not every queue needs to be a pipeline.

Consider the Backlog - the customer is placing the prioritized list of features in this column. The cards in this column exist so that the analysts will know what work needs to be done - not because any work needs to be done in this column, explicitly.

The Delivery column may not need a pipeline, either. If the delivery process is composed of sending an installer package to the customer, then there is no real work to be done in this column aside from sending that package. However, if there is some specific integration work (say, changing a web.config file) that needs to be done, we could include a WIP and Done pipeline for delivery. For this simple example, we'll say that there is no configuration change needed. Let's assume that the installation package changes all the needed configuration files based on user input.

With all of this in mind, we can apply our WIP and Done pipeline to the Analysis and Customer Acceptance columns.

Alternative Visualizations

There are other ways of visualizing WIP vs Done cards for a given column, of course. For example, you could forgo the pipeline idea and simply add a queue between Analysis and Development. This may require you to adjust your order points and limits between analysis and development - at least, where those order points and limits apply (Analysis vs. the queue between Analysis and Development). You could also split the column horizontally instead of vertically, creating either the top or bottom half as the WIP with the other being Done. Or you could just note the status on the kanban card itself. I'm sure there are a number of other ways to show this information, as well. The point is that the pipeline visualization that I'm showing here is not the only way. Find what works for you.

Where Do We Go From Here?

We have another tool in our belt, now. If need be, we can visualize the WIP vs work Done for a given queue, allowing us to see when we can pull work forward. But, like all tools in our belt, it is not *the* answer and this additional visualization may not be necessary for your team.

If your team is small or has great communication, you may not need any special distinctions to show work in progress vs work ready to be pulled. You could rely on the daily standup for people to report when they are done, or do it ad-hoc - when someone finished a card, they announce it to the team immediately. In a larger team, though, it may become necessary to have some visual distinction between work in progress and work completed for a given column.

Small teams might get away without it; larger teams might need it; its up to your team to decide. The key is not to apply these ideas as blanket rules that must be followed, but to allow the individual project and team to decide what is right for them. And as always, allow the process to change when it needs to. Always inspect, adapt and continuously improve.

In Part 1 of Kanban in Software Development, I introduced the concepts of kanban boards and pipelines. I also showed a very simple example of creating a pipeline for our development process. However, there were some obvious limitations in what I showed. The reality of software development is much more complicated than three simple steps (Analysis, Development and Testing). A software development team or company is going to have more to do than just these things, and there is usually a need for some team members to be cross-functional. Some team members will have to do development and testing, or analysis and development, or documentation and delivery, or any other combination of steps involved in creating software. In this post, I'll address these issues, creating a more complete pipeline and finishing our kanban board.

Working In A More Complete Pipeline

To create a more complete kanban board, we need more than just a three step pipeline. We need to allow for a fully functional team - developers, analysts, testers, technical writers, and others. We also need to allow the different team members to work on different parts of the system, as work is needed. The end goal is to enable the system to flow through the process and to ensure the work is "done done" before it goes to the customers.

For a more complete software development pipeline, let's use the following columns: Backlog, Analysis, Development, Documentation, Testing, Customer Acceptance, and Delivery. We can put together a pipeline diagram that follows these steps.

Now, let's assume that we have the following team (ignoring project managers, customer representatives, etc, for now): 6 developers, 2 business analysts, 3 test automation engineers, and 3 technical writers. Given this team, we would not be able to make a feature flow through this pipeline. We would have developers sitting around, waiting for work from the analysts. And, our documentation writers and test lab people would probably be pulling his hair out from boredom then pulling their hair out from too much work, in an unbreakable cycle.

Fortunately, we can account for the team makeup and the potential bottlenecks by introducing the concept of order points and limits that we saw in the grocery store, and by creating queues and multiple pipelines to be worked.

The Real World - Multiple Pipelines and Limits

In most software development efforts, it's unreasonable to expect an entire team to work on only one feature at a time. Most development managers want to maximize the throughput of development by working on tasks in parallel. Given this desire, we'll divide up our developers into three teams of two developers each. This development team division should allow three features to be developed at the same time. Since we want to work on three features at a time, we will need three pipelines for work.

These three pipelines constitute our first limit - we can have a maximum of three features in development at any given time. This limit is noted by putting a "3" in the upper right hand corner of the development column header, as shown above.

Having three development pipelines puts us in a tricky situation, though. We have very few business analysts compared to developers - even with our developer teams, we have less than one analyst per developer team. Fortunately, development work often takes longer than analysis. We can use this knowledge to our advantage and let the analysts work slightly ahead of the developers, creating a queue of work to be done.

Handling Bottlenecks - Queues, Limits and Order Points

If all three development teams happen to finish at the same time and need more work, we would need a minimum of three features that are ready to be worked. However, since we know that requirements change regularly, we don't want to get too far ahead of the developers. With this in mind, we can introduce an Analysis order point of three (the number of development teams) and limit of five. This gives the analysts the ability to work ahead of the developers and also makes them responsible for keeping work available for developers.

For an order point we'll introduce the number into the upper left hand corner of a column header, and we'll continue using the upper right hand corner to specify our limits.

Since we are now dealing with potentially more work in analysis than development, it seems we would have to increase our pipelines to five. This is not desirable, though, since we only have three development teams. What we will do, instead, is restructure the pipeline and turn the analysis column into a queue.

Since Analysis is now a queue, there does not seems to be a need for Backlog to be a step in a pipeline, either. The backlog is simply the list of features that the customer is expecting to be done next. Therefore, we will also turn the backlog into a queue - this time, with a priority (top of the board is highest priority) allowing the customer to tell us which specific feature should be worked on next. We will also want to keep the backlog column limited, to prevent the team from having too much information to think about. Since we only have two analysts on our team, it seems appropriate to keep at least two features in the backlog column at all times. With this in mind, we can safely set a backlog order point of two and limit of five (enough to keep the analysis column full).

The changing of Backlog into a queue has no made our Development column the first step in our actual pipeline.

Completing The Pipeline - Aggregate Limits and Work Cells

The next few column - Documentation and Testing - both have an easy amount of team members to deal with. There are three technical writers on our team, and three testing personnel. This distribution lets us keep the pipeline in tact between Development, Documentation and Testing, allowing us to set the same limit as Development (three).

With all three of these columns having the same limits, and with each column being properly staffed so that three pipelines could run at the same time, it makes sense to combine the team members from these three columns into a single workcell. If we consider all four team members (2 developers, 1 tech writer, 1 tester) as a team and allow them to work together as a team, we can aggregate the limits of the columns in the pipeline. We can also create a consolidated name for this pipeline - work in progress.

This consolidation can be shown by creating a parent / child header with the limit shown in the parent. We will also use a dashed lines between the columns of the pipelines. The combination of these two details will show us that we are dealing with a pipeline, and how many pipelines are allowed to flow, simultaneously.

By aggregating the pipeline limits, we allow the workcell for a given feature to focus exclusively on that feature. The development staff, technical writers, and test lab personnel will all be working on the same feature, allowing them to more easily share information between themselves. This will prevent the team members from having to switch back and forth between subjects, reducing cognitive load and allowing for greater quality to be attained in the individual feature.

Completing The Kanban Board - Customer Acceptance and Delivery

If the customer is on-site (co-located with the team or close enough to meet every day), then it may be possible to aggregate the Customer Acceptance process into the Work In Progress pipeline. If you can do this, you should. Getting immediate feedback from the customer, while work is still fresh in you mind, is critical to the quality and success of that work. What's more - you may not need this explicit column if you have the customer working with you every day.

Unfortunately, we don't always have the luxury of on-site customers. We may only have the customer around at specific timer intervals or only on request. In either of these cases, we have to account for the bottleneck of Customer Acceptance.  Additionally, the customer may not want to get delivery of individual features. They may want to wait for a quarterly release, or some other regularly scheduled release. Once again, we have to account for this bottleneck. We can apply the same principles that we used for Backlog and Analysis here, and create some appropriate order points limits.

Let's say that our customer wants to do Customer Acceptance testing once a week and wants a Delivery once every calendar quarter. To accommodate this, we can set the Customer Acceptance column as a queue with a 1 week limit. We can also set the Delivery column as a queue with a 1 quarter limit. Setting these limits puts a direct responsibility on the customer. If they cannot pull the features through these columns on or before the specified limits, then the entire system could grind to a halt. It's important to educate the customer in what they are committing to, and ensure that they can fulfill their obligations. This fulfillment may mean that they assign more than one person or group of people to the task of Customer Acceptance and accepting Delivery. Whatever the solution is, the customer has responsibilities to meet.

With Customer Acceptance and Delivery specified, our completed kanban board would look like this:

Put It In Action - A Kanban Process Is Never Truly "Complete"

Now that we have clearly defined our Kanban process and visualized it through our Kanban board, the next steps is to start using it! Start working through this process, pull value from the end of the process and allow work to flow through the pipeline. Respect the order points and limits, and above all, look for problems in the process and fix the process when you need to.

Perhaps you have too few or too many items in one of your queues - adjust the limits and order points. Perhaps the test lab is now a bottleneck - take them out of the pipeline and change them to a queue. There are many possible problems and many possible solutions. The key is to be aware - inspect and adapt and hold regular retrospectives - kaizen your process and continuously improve.

Where Do We Go From Here?

No process is perfect. Anyone that tells you otherwise is selling something. A customer may find a bug during Customer Acceptance; an issue makes it past all of our QA processes and is found out in production; or there's a feature that has a sudden priority over any other feature currently being worked. But our current kanban process doesn't address the natural problems that occur during a software package's lifetime. To address this properly, we'll need to employ the lean tools of Andon and Jidoka, and we'll also need to decide how to visualize this on our kanban board. These issues, and possibly more, will be addressed in another installment of Kanban in Software Development.

In the world of Scrum, XP and other forms of Agile software development, many teams use visual control systems to outline the various steps that software goes through during development. These boards are known by various names -  Scrum boards, card-boards, task-boards, swim lanes, and probably a few other names that I'm not aware of at the moment. Whatever you call it and whatever agile creed you live by, the primary use of these boards is the same all around - information radiation. Anyone can look at the board at any given moment in time, and know where in the development process a given card is. If you're doing iterations or sprints, the board also tells you where you are in that time box - if you're ahead, behind, or on time. The far left column - the iteration backlog - will be filled with cards at the beginning of the iteration. Over the coarse of the iteration, cards will be pushed through each of the columns until they are in the done column. The goal is for every card to be "done done" by the end of the iteration, with a set of features that is now deliverable to the customer.

Kanban Boards

Visual management tools are very common in agile shops, and for good reason - they work very well. It should come as no surprise, then, that kanban software development also employs various visual management tools, including a kanban board.

On the surface, there isn't much difference between an average task board and a kanban board. Each of these boards has various columns that represent the stages that a card needs to go through before it is considered done. The real difference in a kanban board, is not the board itself. The board is just a visual indicator, the same as any task board, and the intention is still to get the cards to the "done done" state - that is, delivered to the customer so that they can use the features from that card. The real difference is how the process is approached - by pulling value through the system.

Kanban Pipelines

When dealing with a kanban process and visualizing it into a kanban board, the various steps that a card goes through is often called a pipeline. A single card starts at one end of the pipe and flows through to the other end. This flow is enabled via the pull system that happens at the end of the pipe.

In the grocery store example from my previous post on pulling value, the pipeline would would likely include the store shelf, the store back room, the warehouse, the supplier and the product creator.

A Simple Software Development Pipeline

A software development pipeline works the same way as the grocery store pipeline. In this case, though, the product flowing through the pipe is likely to be a feature of the software package.

Consider the following three columns in a simple pipeline for software development: Analysis, Development and Testing. When a customer requests a given feature for a software product, they want to pull that feature out of testing so that they can start using it.

Once that feature has been moved out of Testing and the customer is ready to pull the next feature out, there isn't anything to pull. At this point, the Testing people would then try to pull the next feature out of Development.

And the same pull happens from Analysis to Development.

In the end, we have created a pipeline for how our development process works. The work that is done flows through that pipeline based on how often the customer wants to pull features out. As one feature exits the pipeline, another feature can be added into the pipeline.

The key to all of this is, again, pulling the features through the system.

Where Do We Go From Here?

I've briefly shown how we can take three steps and produce a simple pipeline for work to flow through. There is more to software development than just Analysis, Development and Testing, though. We also have to consider team size and makeup, parallel work, and other constraints. In the next entry of Kanban in Software Development, we'll flesh out a more complete pipeline and take the next steps to show how to complete our Kanban board, enabling our customers to pull features out of our development process.

Before I start talking about how our team is going about our implementations of Lean and Kanban, I wanted to start by outlining my current understanding of what kanban is. I'm hoping that this will set the ground work for the rest of my Adventures in Lean series, and implementing Kanban in a Scrum shop.

Groceries, On And Off The Shelf

The next time you're in a grocery store looking at a shelf stocked full of products, take note of the little stickers that adorn the shelf just below where the product is placed. This sticker is usually crammed full of information - some of it useful to you, the consumer, and some of it not useful to you. So why is all of that information there? Who is getting any value out of that information? The answer is surprisingly simple - the people that stock the shelves. When the supply of good on the shelf is running low or is empty the information on those stickers tells the stock-persons what to put back on that shelf. This person is then able to create a pick-list of goods to retrieve from the back of the store so that they can restock the shelves.

Now consider an extended example of more than just the grocery store. When you, the consumer, take an item off the shelf, you have created room for one more item to be placed on the shelf. The stock-person sees the sticker in that space, goes to the back of the store to retrieve the item and places it on the shelf where the space is. At this point, there is another person paying attention to how much is actually in storage in the back of the store. When the stock-person grabs an item from the back of the store, the back-room person sees that space and determines whether or not they need to order more of those items from the warehouse. When the back-room person orders items from the warehouse, the warehouse workers check to see if they need order more of those items from the supplier. When the warehouse orders those items from the supplier, the supplier checks to see if ... and on and on until we finally get to the point where the products are being made - the manufacturer or farm or whatever the ultimate source of supply actually is.

What Is Kanban?

This simple act of seeing that a space on a shelf needs to be filled, and filling it, is the core of what kanban is. Quite literally, a kanban is a sign or signal. In more conventional terms, though, it is a sign for action that must be taken to replenish the empty slots on the shelf of the grocery store, the back room, the warehouse, etc.

From Wikipedia:

"The Japanese word kanban (pronounced [kambaɴ]) is a common everyday term meaning "signboard" or "billboard" and utterly lacks the specialized meaning that this loanword has acquired in English. According to Taiichi Ohno, the man credited with developing JIT, kanban is a means through which JIT is achieved."

When a consumer takes an item off the shelf, the process of pulling products - pulling value, really - from the upstream suppliers begins. It's the pull of kanban that makes it such a powerful system. We let the customer pull the items they want which drives the rest of the pull system. If no one takes the item off the shelf, there is no need for the stock-person to put another item on the shelf, and no need for the store to order more from the warehouse, etc. Think of it as a form of Demand-And-Supply instead of Supply-And-Demand. The customer demands a product, the grocery store supplies it. The store demands the product, the warehouse supplies it, and on and on up the stream.

What Is An Order Point?

When a space is empty on a shelf, a stock-person may not immediately fill it. In fact, the stock person may wait until there are several spaces on that part of the shelf before filling them. On this shelf, the maximum number of empty spaces allowed is the order point. For example, if the shelf has three empty spaces and the order point is set at three, the stock-person knows that they need to replace the items on the shelf soon. If the number of empty spaces goes up to four, the stock-person knows that they need to replace the items on the shelf now.

Order points can be specified in a few basic ways:

• Maximum number of empty spaces needing to be filled
• or Minimum number of products on the shelf

I'm sure there are other ways, as well. These are the two that I've seen the most often, though.

Each area of the grocery store is going to have it's own set of order points. The shelves that consumers pull from will probably have a very low order point - may two or three items missing. The storage area of the store will probably have a larger order point - two or three cases missing. On up the supply stream, the warehouse may have a larger order point of two or three pallets, and on and up the stream the order point may grow larger.

In the grocery store, the kanbans that adorn the shelves often have the order point printed directly on them. This gives a stock-person the knowledge they need, when they need it, and prevents them from having to remember the order points for all the products in the store.

What Is A Stock Limit?

When a shelf at the grocery store is full of products - there is no space left to put anything else - that shelf is full. However, the individual products on that shelf may not be at their stock limit. A stock limit is the maximum number of a given item that can be stocked at a given time - wether that stock is on the shelf or is in the storage area of the store, or wherever it is. If a shelf has a stock limit of five, for a given product, then there should be no more than five of that product on the shelf. Both the shelf and storage area have their own stock limits. It's these stock limits that prevent the store from being overrun with too many of one product, and not enough of another product. 

Like order points, the kanbans that adorn grocery store shelves often have the stock limit printed directly on them. This tells the stock-person when to stop putting items on the shelf without them having to remember the limits for all the products in the store.

How Does This Apply To Software?

I'm not going to give away the farm just yet. Moving forward, though, I will be using the terms Kanban, Order Point, and Stock Limit (or just Limit). I wanted to get these core terms defined up front, so that they will make sense in the context of what is still to come.

Stay tuned for the next entry in my Adventures In Lean!

Finding Classes With Resharper

It's no secret that I'm a huge fan of Resharper. It rocks. I don't like to code without it. One of the many features that I love is the Ctl-N shortcut to find a class. Resharper gives you this handy-dandy little search box:

What I really love about this box is the ability to not know the entire class name when searching. If I know my class involves the word "Super" and "Sexy", I can type the letters "SS" and the search box will pull up any class with matching uppercase letters.

The same holds true for lowercase letters. I can do "SupSeV" and get results just matching those Upper/lower combinations.

BDD Context Specifications Have Long Strange Names

It's also no secret that I'm a fan of BDD and Context/Specifications. I love the language oriented nature of context specifications and how it's easy for me to see what the behavior of the system is supposed to be, in any given context. I've been using BDD style syntax for many months now, and have amassed quite a collection of Context/Specification tests in my current code - especially with 4 other developers using BDD syntax. After having done several hundred tests in this manner, I've found that there is a pretty significant disconnect between how I use SpecUnit.NET and how Resharper's class finder works - the names of my specification classes. Look at this specification class name for example:

How am I supposed to search for this class name? I can't remember all those words, none of them are capitalized, and all those underscores are probably going to throw Resharper off in my search string.

Organizing Context/Specification Classes By Parent Class/File Name

To combat this problem, what I've started doing recently is throwing in the use of a parent specification class with the same name as the specification file that I'm working in. Since our team has standardized on the "Specs" suffix for all of our BDD tests, I know that a file name of "ValidationSpecs.cs" will have a class called "ValidationSpecs". In the file itself, my specs will be subclasses, like this:

With the file name ValidationSpecs and the parent class ValidationSpecs, I now have much fewer words to remember and a much greater chance that I'll be able to use Resharper's class finder feature. All I need to know that I'm looking for the tests that deal with validation, so by our naming convention, I can type in "VS" or "ValSpecs" and get the list back that I want:

In the last six months, my team has undergone some very radical changes and has turned into a full blown Agile team. I'm very happy with our success and I consider this team to be the shining example in our company, at the moment.

Now, in keeping with the tradition of this team in changing at least one thing every few weeks, we are about to embark on a new journey in our project management processes: Lean Software Development.

Starting at 1pm, Central Time, today, my team will be kicking off the following processes:

• Kanban
• Release Per Feature
• Just In Time Retrospectives

I am going to be posting a series of entries on each of these specific items over the next few weeks, and will most likely keep this post updated as the index of entries for a while. So stay tuned for a whirlwind of opinionated posts on our next great experiment!

Table Of Contents For Adventures In Lean

Here are the articles that I have written or am writing for my Adventures In Lean series:

Kanban

1. Kanban - Pulling Value From The Supplier - In this post, I am laying the foundation of what Kanban is along with a couple of other important terms that I will be using throughout the series. Kanban is only one of many tools, techniques and philosophies found in lean, though. Trying to sell kanban as lean would be like selling a steering wheel as an entire car - you're only getting part of what you really need.
   
2. Kanban in Software Development
   1. Part 1: Introducing Kanban Boards and Pipelines
   2. Part 2: Completing the Kanban Board with Queues, Order Points and Limits
   3. Part 2.5: A Variation on Queues - Pipelines for WIP and Done
   4. Part 3: Andon and Jidoka - Handling Bugs and Emergency Fixes in Kanban
3. Our Kanban Board and Process 
   
4. Release Per Feature - Delivering Value As Soon As Possible
   
5. Just In Time Retrospectives - Fixing Problems As Problems Occur

There are likely to be other articles added to this list as well. Please check back now and then to see what has been added!

A coworker recently asked if we should always abstract every object into an interface in order to fulfill the Dependency Inversion Principle (DIP). The question stunned me at first, honestly. I knew in my head that this was a bad idea - abstracting into interfaces for the sake of abstraction leads down the path of needless complexity. However, I wasn't able to clearly answer his question with specific examples of when you would not want to do this, at the time. I've been thinking about this for a few days now and I think I have a good, albeit very long winded, answer.

Before the question is answered, though, we need to step back and look at what DIP is all about. I've previously shown how to implement DIP and talked about why it's beneficial, so I won't be repeating that here. Rather, I want to talk about the language that describes DIP and what it really means.

Robert Martin's original definition of DIP is this:

A. High level modules should not depend upon low level modules. Both should depend upon abstractions.
B. Abstractions should not depend upon details. Details should depend upon abstractions.

The word 'abstraction' is used three times in the definition for DIP. So, in order to understand DIP, we have to first understand some of the basics of Abstraction.

Some Background On Abstraction

From Wikipedia (emphasis mine):

"In computer science, abstraction is a mechanism and practice to reduce and factor out details so that one can focus on a few concepts at a time"

I can't say it any better than this.

Abstraction can very directly lead to a system that is more understandable by helping us ignore the detail and implementation specifics, allowing us to focus on something at a higher level. This reduction in cognitive load can benefit someone that is reading the code by not forcing them to know the detail immediately. Additionally, abstraction is a form of encapsulation or information hiding, which again helps us to reduce cognitive load and produce better systems. From Wikipedia's entry on Information Hiding:

"In computer science, the principle of information hiding is the hiding of design decisions in a computer program that are most likely to change, thus protecting other parts of the program from change if the design decision is changed."

At the heart of abstraction and information hiding, we find the ability to change the system. The ability to change is an absolute requirement in software development and produces good design that is easier to work with, modify, and put back together as needed. The inability to change is directly called "bad design" by Robert Martin, in the DIP article.

Applying Abstraction And Encapsulation To DIP

So how does our knowledge of abstraction and information hiding play into DIP?

First and foremost, DIP never states that we should depend on explicit interfaces. Yes, in C# we have an explicit Interface as a form of abstraction. It is a separation of the implementation detail from the publicly available methods, properties, etc, of a class. Some languages, such as C++, don't have an explicit construct for interfaces, though. From Robert Martin's original DIP article, again:

"In C++ however, there is no separation between interface and implementation. Rather, in C++, the separation is between the definition of the class and the definition of its member functions."

A String As An Abstraction

Abstraction does always mean explicit interface constructs, as evidenced in C++. Nor does it always mean an abstract base class, which we also have available in .NET. In fact, languages such as Ruby don't really need either of these constructs. The duck-type nature of Ruby allows an implementation to be replaced at any point, without any special constructs. In .NET, though, there are a number of abstraction forms that we can rely on, explicitly. We have the obvious interfaces and base classes (abstract or not) - but we also have constructs like delegates and lambda expressions, and even the simple types that are built into the base class library.

Let's look at a simple string to illustrate abstraction. As I said in my SOLID presentation at ADNUG, we can invert our dependency on database connection information.  Rather than putting a connection string directly into our code that calls the database, we can use the string as our dependency and our abstraction. All we need to do is follow the basic DIP principle and provide the string as a parameter to the class that calls the database. We certainly don't need (or want, for that matter) to introduce a new interface or base class at this point. Our abstraction is simple enough to use a common type found in the .NET framework.

Other Forms Of Abstraction

Even if we are talking about an object, who says that the interface we are depending on has to be an explicit interface construct or base class? When I write a Domain Service that uses an Entity from my Domain, I don't create an explicit interface for that Entity. Rather, I use the Entity's inherent interface - it's public methods, properties, etc.

I also use delegates on a regular basis. By specifying my abstraction as a delegate, I can further decouple the depending object from the dependant code that it needs to call. I'd be willing to bet you have used delegates as abstractions as well. Have you ever created an event handler for something like a button click? There's a delegate's abstraction at work.

Abstract Judgement

The point is, there is not always a need to introduce an explicit interface or base class when inverting our dependencies. We still need to apply dependency inversion and provide our implementation as a constructor parameter (or setter, though I don't like setter injection). But, that dependency doesn't have to be anything more than the interface inherent to the object, or a simple type found in .NET.

You do have to be careful when making the call to not use an explicit abstraction with DIP, though. You can quickly turn your system into a ball of mud if you rely on a concrete class that is not intention revealing or well encapsulated to begin with. At the same time, too many abstractions can lead to needless complexity and make it very difficult to see the big picture of a system. Too few abstractions, though, will certainly lead to a rigid, immobile design that is hard to change. All of these problems are equally vicious - and I've been bitten by all of them in recent months.  It takes good judgement calls to determine when you do and do not need an explicit abstraction for a dependency. Unfortunately, good judgement comes from experience and experience comes from bad judgement. Don't be afraid to make bad decisions - make a decision, just be sure you can reverse that decision as easily as possible.

I was reviewing my calendar for the upcoming month of October, and it looks like I'm going to have a crazy-fun month!

October 3rd & 4th: Pablo's Day(s) of TDD

This is going to be a great couple of days of learning, and hopefully contributing, to spread 'the good news' of TDD.

October 7th: AgileAustin montly meeting

All about test automation! A subject I'm very interested in, recently.

October 13th: Austin .NET User Group (ADNUG)

Presentation by ... ME!

I'll be giving my SOLID Software Principles presentation, talking about why we want SOLID software, as well as how to achieve it.

October 30th - November 2nd: KaizenConf: Continuous Improvement in Software Development

Otherwise known as an Alt.NET Conference. This is going to be a great weekend of learning from the best and brightest minds in the Alt.NET community, as well as Tom and Mary Poppendieck!

... October is going to be a whirlwind of learning, for sure.

After some additional Twitter discussion with Jimmy last night, I realized that my previous response had neglected to distinguish between two very important contexts - new code and legacy code.

Considering a legacy system (as defined by Michael Feathers), I'm 100% on board with everything Jimmy is talking about and all of the techniques, troubles, and diminishing returns that he identifies.

I still maintain that 100% coverage should be the default result of true TDD when writing new code - and I apply this philosophy when writing new code inside of a legacy system. Write a test to prove you need the new code that you are writing... unfortunately, the distinction between new and legacy code does get blurry and can be difficult to navigate when working in a legacy system.

For more info on legacy code, in this context, and how to deal with it:

• Working Effectively With Legacy Code - the book by Michael Feathers
• Working Effectively With Legacy Code - Article on ObjectMentor by Michael Feathers (consider it the cheat-sheet for the book)
• Jimmy Bogard - He's done a "Legacy Code" presentation several times, based on his own use of the Michael Feathers techniques.

There's been a lot of recent talk about what is "done" in the Lean / Agile development communities, and the primary focus of these discussions has been focused on individual features or stories and the need to get them completely "done" (dev, test, acceptance test, documentation, delivered to customer) before they can be considered done. I wholeheartedly agree with this philosophy, to the point where I active introduce painful elements of software development to my team members, because the work must be done before we can deliver to the customer.

All that being said - I think we're missing out on some potential benefit by not applying "done" at various and different levels of our software development projects. For example, the concept of "done" can be applied to an iteration. What constitutes an iteration being "done"? It's certainly not just the time-box of 2 weeks, 1 month, or whatever our iteration length is defined as. However, that time box is still important. We don't want arbitrary lengths of iterations.

So how do we know when an iteration is completely "done" versus just being over? My initial thoughts revolve around a pass/fail checklist, similar to the swim lanes (or kanban board, if you want to call it that) that our stories go through. At this point, I would likely include the following:

• Iteration length passed
• All stories in iteration "done"
• Code reviewed (possibly part of story being "done")
• Software is in a stable, working state
• Software acceptance tested by customer (or customer representative / SME)
• Retrospective held

There's probably some additional items to include here - this is just my initial idea list.

Like the user story "done" criteria, an iteration cannot be considered "done" until all of the items in this list have been checked off. We don't give partial credit for a story making it to "in test", when there are 2 more columns to move through - the story is not done until it's completely "done". The same should apply to the iteration.

Why bother making an official "done" list for iterations? For the same simple reason we do this for user stories - transparency and visibility into bottlenecks and problems. It's easy for a team to ignore problems like broken software at the end of an iteration - "oh, it's just XYZ... i'll take care of that tomorrow" - when we don't have the same accountability as we do in our stories. By making the iteration a pass/fail set, we expose those problems for the world to see. This exposure is a great motivational tool - who wants to be caught with an un-"done" iteration because of a "simple" bug?

Do you and your team have an iteration "done" list? Do you even have a user story "done" list? It's all about exposing weakness and waste, then eliminating it and creating the official iteration "done" list is just one more step along the path. Don't stop there. What other processes - higher or lower level - can we standardize and improve?

Derick’s (brand new, just thought of it, but is now elevated to ‘mantra’ status for me) golden rule of Acceptance Criteria:

If it’s not usable by every team member, it’s not Acceptance Criteria.

And I do mean every team member - Customers, Testers, Tech Writers, BAs, Developers, UX Designers, and anyone else on your team.

You can specifying the technical or UI details in the story’s detail, but it’s not acceptance criteria – it’s Technical Criteria, or UI Criteria, or Test Automation Criteria, or … etc. Generally speaking, don't include these alternate criteria in the story detail – let the specific team members determine their specific criteria and record it how they need to (through unit test for devs, interaction design mockups for ui peeps, etc).

The entire team – BAs, Devs, UX, Testers, Tech Writers, Customers and Management (especially customers and management) – all need to understand that the first few iterations of a team that is trying to convert to an agile methodology, will be slow and painful. You won’t get as much done as you think you should. You’ll run into constant problems, questions, points of clarification, and unknowns.

Iterative development processes expose every single problem that your team never knew it had, and more – and if you’re honest and working with integrity, you won’t whitewash the problems and try to hide them; rather, you’ll embrace the transparency that iterations create and use it to drive improvement in your team.

Converting to Agile is painful and it’s embarrassing at time. There's no question that it’s difficult to convert – especially if the team has any significant habits or experience from previous projects (and who doesn’t, aside from entry level people). Work through the pain, solve the problems one at a time and continue to improve the process. Eventually, the team will find it’s groove and productivity will increase. Over time, if the team truly is honest with itself and is continuously solving problem, productivity should skyrocket and you’ll wonder how you ever worked without iterative / agile methodologies.

The key is - don’t expect it to be perfect… ever… take one step at a time and never stop moving forward.

In addition to our 'Code Review Challenge' that I discussed recently, we have been using another retrospective game - 'Name That Standard'.

The idea is fairly simple. At the beginning of any retrospective, we go around the room and ask each team member to list out a standard that we are using in our project. This can be any standard that anyone feels the team is using or needs to be using, including coding conventions, architectural patterns, business and project management processes, communication means, etc. The intention is similar to the intent of 'The Code Review Challenge' - socialization of the the system. Only in this case, we are applying the term 'the system' at a much higher level - our organizational and project management / implementation system as a whole.

Some thoughts on running 'Name That Standard':

• Require not only naming of the standard, but accurate description of the standard and an example of where it is used. Alternatively, you may want to list a brief description out on a projector or whiteboard and see who can name the standard being described.
  
• Encourage the team to think about the team as a whole, not just their area of responsibility.
  
• Encourage the team to list standards that they don't understand, so that the rest of the team can chime in with assistance and help the team member in question grow
  
• Don't just let people yell out standards. go around the room, one person at a time. we got a little chaotic at first, and it was hard to know who said what.
  
• Encourage people to ask questions like 'i thought we were using this standard, but i saw someone else doing that standard'
  
• Encourage the standards to become more and more low level and detailed, over time (over multiple iterations)
  
• Encourage the standards to evolve and change for the better, over time (over multiple iterations)

In our implementations of this game, we've managed to get a fairly good list of standards written down and also resolved some questions on what standards we actually are using. For example, we had 3 dynamic mocking frameworks in our code base - Rhino Mocks, Moq, and a home grown one that we lovingly called 'BrandoMocks' (named for our team member, Brandon, who wrote it for fun one night). The end result of that discussion was to throw out Moq and BrandoMocks, adopting Rhino Mocks as our standard for this project.

Having done 'Name That Standard' in two consecutive iteration retrospectives, I think I can say that it has been very successful. The team as a whole is becoming more and more aware of the standards that we are using, and the individual members are beginning to contribute more and more to the standards.

With each passing iteration, and each new technique for creating collective ownership and socializing the system, I see the team coming together and really forming a team vs. a bunch of individual developers working on the same system.

Test Driven Development Is To Unit Testing As Interaction Design (IxD) Is To Accidental Design

One of the major problems with writing unit tests after the code is that it is very natural to write tests that prove the code works the way it was written, instead of the way it should work. Writing code test-first (via whatever flavor of Test Driven Development) flips that scenario right side up. TDD helps to ensure that code is written to work according to specifications and not the other way around.

Similarly, designing and/or implementing a UI after a behavior or process has been coded is a likely to result in a UI that fits the code model and not a model that fits the needed interaction and workflow. This situation must also be flipped right side up - interaction design should be done before, or at least in parallel, with coding. It cannot be left to accidental or incidental happenstance - interaction design must occur with proper interaction patterns and practices in mind.

Overlapping IxD and TDD

To overlap interaction design and test driven development, there are a few key words that need to be borrowed from interaction design. Fortunately, they easily fit within TDD development techniques and philosophies.

Epistemic work is exploratory in nature, or a process of trial and error through research. TDD and interaction design sketching are both epistemic. TDD explores API possibilities and allows easy trial and error to find the simplest implementation for what you need at the time. UI design sketches also allow you to quickly explore interaction designs - whether it's a white board, pencil and paper, graphics design software, or even quick-hack forms layout in IDEs. You can quickly and easily throw away a bad API in TDD and you can quickly and easily throw away a bad UI/Interaction design when you have nothing more than pencil sketches or white board drawings.

Pragmatic work is very structured and step-by-step in it's nature, implementing patterns and practices to fulfill what is needed at the moment. Implementing Code after writing unit test and implementing UI after designing around constraints are both pragmatic. TDD is pragmatic in that you only implement what is needed to properly pass the tests that have already been written. Similarly, with previously designed interactions and UI elements, implementation can be easily limited to what is needed for the UI.

With epistemic and pragmatic work covering both interaction design and test driven development, it seems that they are a natural pair. An analysis of a user story and it's acceptance criteria will create the unit tests that we need. At the same time, the same analysis can be applied to interaction designs. Additionally, a strong understanding of how a UI will look can have a profound impact on the code that is written, and vice-versa. Therefore, it is natural that interaction design and test driven development are done at just-in-time intervals - before the real work is implemented in the real code and UI platform.

IxD As Part of "Done"

Despite interaction design being a required part of UI development, not all user stories require a UI. Interaction design may not fit into a swim lane board or be part of every story's "done" criteria. However, interaction design will always be done for any story that does have a UI - it may simply be an accidental or incidental part of the software development process for a team, though. If it's safe to assume that the work will be done, then it is the team's responsibility to ensure that it is done correctly. Don't let interaction design happen accidentally or incidentally in the development process. Set a standard of always including interaction design in the development process, the same way Model View Presenter/Controller is a part of development.

Scott Bellware (who finally started blogging again), is posting a series on Sustaining Capacity in Maturing Agile Software Teams. It's worth a read, and may be familiar territory to anyone who has done Agile and been reading about Lean.

In the 3rd Post - Recognizing Entropy - he talks specifically about synchronizing the development testing and QA testing efforts:

Development and QA Test Synchronization. Without greater synchronization between development and QA testing, valuable input often comes too late in a development or release cycle to be effective. Test design and test architecture are valuable inputs to development, and software design and architecture are valuable inputs to QA testing. Teams often loose unrealized capacity by not pursuing means to do more development and testing in parallel.

This one really hit home for me, and is a shining beacon to the problems that my current team is going to face, soon. We had our first iteration planning and kickoff meeting yesterday, and the whole team was involved - except the testers from the QA department. For reasons that I have no control over (and honestly - they are legitimate business reasons), the QA department is not coming on to the project for another 2 months (at least).  Today, the team leadership was discussing how the QA personnel would create their test scenarios, and I brought up the point that creating the test scenarios after-the-fact (after the iterations are done) is going to be very problematic.

If a dev team is working from User Stories, and those User Stories are divided along various aspects of behavior, it is very likely that a single screen, workflow, or 'function' of a system is going to be divided into multiple user stories. When the QA team is involved with testing in parallel with the development team, this is not an issue because the QA team will be updating their test scenarios in each iteration. This allows them to keep the scenarios for a given screen, workflow, or 'function' of the system, growing over time. However, since our QA team is going to come into the project at least 2 months after the development team has started coding and testing, the QA team would have to sift through all of the current and previous iteration stories, try and aggregate them down into the screen, workflow, or 'function' of the system, and build test scenarios from there. If there is even a modest number of stories to sift through, this would be a daunting task for the QA team - they would be in a 'catch-up' mode from the beginning of their QA cycle, likely through the end of the project.

As a result of the QA team being unable to join our project immediately, we are forced to modify our entire development process and account for the needs of the QA team, after the fact. Essentially, we are creating additional swim-lanes for our stories to travel through, during an iteration:

1. Backlog
2. In Development
3. Developer Testing
4. Ad-Hoc Testing (BA's, Developers, etc. testing the app)
5. QA Functional Spec Documentation
6. Done

Swim-lane #4 and #5 are directly affected by the QA team not being able to enter the project immediately, and are causing the rest of the team to lower it's capacity. We (developers and BA's) are now required to do our own ad-hoc testing during the iterations, instead of letting the QA team take care of that; and we (the developers and BA's) are now required to create functional specifications, based on the user stories that are completed, to hand to the QA team once they are on the project. The functional specification documents will need to be updated for any given User Story, before the ticket is considered complete (the functional spec updates have become an acceptance criteria on every User Story).

End result - because we cannot get the QA team on our project immediately, and ensure that they are testing in parallel to the developers, we as the development team are losing capacity and building entropy from the very first day of the very first iteration.

Here's the format that my team is currently using for User Stories and Acceptance Criteria. These formats are primarily learned from Scott Bellware's training that we have had recently, but is also influenced by the actual project that we are currently working.

User Stories

As a [Role], I want to [Goal], so that [Motivation].

Role: Who is using the system

Goal: What you want to do with the system

Motivation: Why you want to use the system.

I typically see people asking questions about Motivation - why it's important, etc. It's important because it gives the implementer a frame of mind for the behaviour.

Example:

As a Nurse, I want to record patient pain levels, so that I can adjust their medication doses appropriately.

vs.

As a Nurse, I want to record patient pain levels, so that I can show the pain trends with certain types of medication over a period of time.

When I read these two stories, as a developer, I begin to see drastic requirements differences and expect certain Acceptance Criteria based on the requirements differences that I expect to see. If I'm only adjusting their medication level, I probably only want to see what their current pain level is and whether or not I can adjust the level within dosage limitations. If I'm monitoring trends over a long period of time, I will want to record more - dose for period of time, when adjusted and why for the pain level, what other medications were involved in that patient in the same time periods, etc. The motivation can cause a huge difference in how the story is implemented.

Acceptance Criteria

I've noticed that there are two types of Acceptance Criteria: Behavioral (functional) and technical (non-functional). As such, there may be two different formats for a story.

Functional Acceptance Criteria

When [verb], should [verb], should ...

Example:

When recording the patient pain level
  Should represent the level on a scale of 1 to 10 where 1 is no pain and 10 is extreme pain

Non-Functional Acceptance Criteria

Should [technical detail]

or

When [functional criteria], Should [technical detail]

Example:

The pain level recording screen must be usable via a touch screen tablet pc.

or

When recording patient levels on a tablet pc, the data should be cached locally and synchronized to the master database when the nurse docs the tablet pc.

I believe the functional acceptance criteria should be stated in technology and implementation agnostics terms - imagine for a moment that you are going to implement the requirements via building blocks, paper forms, or printed circuit logic. If you have to change the functional acceptance criteria based on the implementation technology, then the criteria is not formatted correctly in the first place.

Avoid joining functional and non-functional acceptance criteria into a single acceptance criteria. It is easy for a developer to lose sight of the behavioral goals when these are mixed, and may prevent a UI/UX specialist from creating the greatest workflow ever.

There are a few examples in The Toyota Way, of American manufacturing companies that were thought to be world class Lean Manufacturers. When representatives from Toyota began working with these companies, though, the result was quite a shake-up. The companies that were thought to be lean, were merely implementing some of the surface level processes of lean manufacturing. They had not realized that there was a root cause of the process that they were implementing.

The Principles, Not The Process

A coworker recently asked me to help out with a presentation on Model-View-Presenter. He had some specific questions about why we would want to use MVP vs. MVC vs. any other UI pattern. Many of the questions centered around various benefits that have been promoted by myself and others out in the developer world via blogs and articles - items such as testability, changeability, flexibility, and other "ilities". My answers to some of these questions surprised me.

Model-View-Presenter is often thought of as a journey and an end in itself. When I first started learning MVP just over a year ago, this is exactly how I saw it. I started with the intention of learning how to separate my core process from the form implementation, with the intention of being able to unit test the core process. I was working under the impression that I could find a way to implement MVP that would truly allow me to swap out my UI - to the point of changing the workflow within the UI. I had the goal in the mind, of being able to write one presenter and have it support WinForms, WebForms, Web Services, and any other client-facing API.

One year and several projects later, I have some good lessons learned. I have implemented various forms of MVP - automagic view injection, manual everything, and a lot of ideas in between. My understanding of MVP is fairly solid, and I am capable of implementing it in WinForms, WebForms, Web Services, and other client-facing API's.

What I have realized recently, is that MVP is not (or, should not be) a journey and a goal in itself. MVP, MVC, and other UI related patterns are actually an effect of other underlying principles. The underlying principles of object oriented development - such as Separation of Concerns, Single Responsibility Principle, Dependency Inversion and others - are the real cause of MVP, MVC, etc. Testability, flexibility, and the other "ilities" are merely side-effects of good design and implementation. And while Testability is a valid goal in itself, seeking it as the primary or only goal will only lead you part way down the path. You will end up like the American manufacturers that thought they were world class lean companies, only to find out that they had barely scratched the surface of lean process.

The Journey, Not The Goal

Despite the success of Toyota as the creators of lean manufacturing, they believe that they are continuously learning about lean, so they can continuously improve. Similarly, despite my many years of software development experience, I now believe that I know little about software development. I have a good knowledge of .NET and general knowledge of Object Oriented Development, but I don't know much about other paradigms like Functional Languages or Duck-Typed languages such as Ruby.

In my professional growth as a developer, I have set goals for myself at various times. Most of these goals were set with the belief that they would make me a great developer. However, I've found that every time I have reached a goal, there is an entirely new world of possibilities ahead and what I thought was a great developer was merely the beginning of a journey. I've discovered that the journey itself is often far more important than what I believed the goal to be.

I'm not advocating that you forget about setting goals and let your career happen as it will - that's a great way to become burned-out, outdated, and irrelevant. What I am advocating is that when we have a goal in mind, we set out with the understanding that the goal itself may only be a small leg on a long journey. In this case, the journey is the process of learning. The journey itself should be viewed as the long term strategy, to the point where you allow the individual goals to come and go as needed. You may not accomplish a specific set goal - but the experience gained while working toward that goal may open up opportunity for other goals to be reached easily.

Learn The Principles Via The Process

All this talk about knowing the underlying principles is great - but how can we understand the principles and philosophies without knowing the process? Unfortunately, I don't know if that's possible for most of us. I'm sure that somewhere out there, someone has read SRP, DI, SoC, and other principles, and as a result began writing code that was modeled in a manner similar to MVP (Martin Fowler, for example). For the majority of us, though, learning the principles is done by first learning the process - study the implementation of the effect to learn the cause.

Conclusions

The journey itself is how we build our experience and understanding. For most of us, that journey begins with a goal of learning a new process. Find a goal or two that you believe will help you be a better developer and learn the processes. Along the journey of reaching for this goal, always keep in mind that there is an underlying principle or philosophy that enables the process. By learning the principles and philosophies behind the process, the process itself becomes a trivial matter - an implementation of the principles and philosophies, and not a goal in itself.

The Toyota Way mentions the concept of Jidoka in chapter 1 (and probably other places that I haven't read yet). On page 6, in the "4 P" diagram, jidoka is described as

"Stop when there is a quality problem"

Wikipedia calls the same concept "Autonomation" and says it may be described as

"intelligent automation" or "automation with a human touch"

and

"At Toyota this usually means that if an abnormal situation arises the machine stops and the worker will stop the production line."

We can apply the same principles in software development in many different ways. One of the more common implementations is the use of Continuous Integration and Automated Testing.

According to the notes I took during A Day Of Bellware, if our CI server says the build is broken, we need to immediately stop working and fix the problem. I've heard this before, and will likely hear it again. However, I never really understood why people would say this. After all, as long as the problem is fixed eventually, isn't that ok? In the training that day, Scott provided a excellent visualization of why we should fix it immediately.

Waste and Rework

To steal Scott's illustrations, consider the following image to be an example of "perfect" software. All of the edges of each module (block) are well defined and it's easy stack new blocks next to existing blocks.

Figure 1. "Perfect" software

Now let's assume that somehwere in the coding process, someone accidentally causes a defect in the software. That defect could be represented by a buldge in one of the lines - as if the module was doing more than it should.

Figure 2. A defect

The individual block that has the defect may not be that bad, at first glance - or when examined on it's own. And on it's own, the defect could be fixed. Jidoka would say that you need to stop immediately and fix the problem.

So, what happens when you don't fix the problem right and and you try to stack another block to the right of the defect? Suddenly you find yourself re-shaping the next block in order to account for the problems in the previous one. Eventually, you may be able to smooth out the issues. Depending on the size of the issue, though, it may take several new modules to completely normalize things.

Figure 3. The effect of a defect

Now, 3 day, 3 weeks, 3 months, or whatever period of time later, you have a significantly larger problem to deal with. Let's go back and fix the original cause of the problem, to start with. What happens to the rest of the code that was warped around the original defect? The warped code in the rest of the system also has to be fixed. Simply removing the original bulge does not mean that the rest of the warps will magically disappear as well.

Figure 4. Removing the original bulge

How much re-work will it take to fix the rest of the system that was warped around the original problem? How much time and effort will be wasted because the original defect was not addressed immediately? These questions can only be answered in the context of your problems. If this defect is one line of code in one module, maybe it's not so bad. But if this defect is an entire module used by many other modules, the time and cost could be huge.

Conclusion: Fix It Now

If the continuous integration server says that our build is broken; or if the customer says "the software does not break, but piece XYZ isn't correct"; or ... pick a problem in your system; we need to respond as quickly as possible, in order to prevent the rest of the system from being warped by the original problem.

Under the assumption that we want to fix a defect as soon as possible - how do we ensure that we know about the problem as soon as possible? Whatever your implementation of this solution is, the solution to this problem comes down to shortening the feedback loop. If you are notified of the problem in 20 minutes vs. 20 days, there will be significantly less damage to the overall system and significantly less re-work and waste caused by the defect.

Myself and 10 other developers in my company went through a day of BDD / TDD training, with Scott Bellware, yesterday. It was a lot of fun, very challenging at times, and covered a lot of topics including an overview of Agile and Behavior Driven Development, all the way down to writing Specification Tests, doing Test Driven Development  and refactoring the model to improve readability, maintainability, flexibility, etc. I took notes via index cards (love that cool-aid) and wanted to share. I don't expect these notes to make sense to everyone. Hopefully it will spark some dialog in someone's mind and cause them to dig further.

First off - the quote of the day.

"Can I be honest with you and say that I've been wanting to touch your keyboard, all day?"

Now for my notes.

User Stories

[Role], [Goal], [Motiviation]

• As a [role], I want to [goal], so that [motivation]
• Example: As a nurse, I want to record a patients vital signs, so that I can determine their medication and care needs
• Motivation is critical - it determines how the development team understands and implements the story. It determines the user experience, how things are integrated, how the software is designed, etc.

Acceptance Criteria

• Acceptance Criteria is used to drive code, not the story, directly
• may change at any point, up to implementation
• is used to drive code design, test design, implementations, etc.
• should be spoken in domain language
• may include non-functional, technical details such as database tables, infrastructure, performance, etc
• All acceptance criteria must be met and tested / verified before a story is considered done

Specification Tests

• Test Fixture per Class is an anti-pattern (on a personal note, this problem bothered me for months before I discovered BDD)
• Context Specification or Behavior Specification testing
• When [verb] then [verb]
  • "When [verb]" is the context
  • "Then [verb]" is an observation of the behavior
• Based on Acceptance Criteria, but not "code-gen'd" from acceptance criteria

Story Estimation

• Agile Poker: uses generalized Fibonacci sequence as order of complexity
  • "?", 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, infinite
• everyone throws their estimate at same time
• if estimates have significant outliers, discussion occurs to understand why, get more detail, etc. and re-throw may happen

Entity Data vs. Aggregate Data

• Entities should never contain aggregate data
• Aggregate data is for reporting and other aggregate needs
• If you need aggregate data to process something, write an SQL query, stored proc, etc. - don't use an ORM like NHibernate
• We don't want a "Customer" entity to need 10,000 "Order" entities, to aggregate data for processing; write a query to aggregate instead
• We don't want to persist data that can be calculated / aggregated, generally (performance issues may override this)

Domain Services

• Can have dependencies on external systems
• are part of domain logic, therefore are in domain model / assembly
• are "Doers" of process that don't fit into entity and entity logic, directly
• coordination of entity logic
• can include calls to data access, logging, etc.

Continuous Integration

• Not just continuous compilation of code
• Full end to end integration of all code, components, databases, services, etc
• Full suite of integration testing including database testing
• Do not allow commits if build is currently broken
• do not allow defects to live - fix immediately, to fix build
• "Defect" is broken software, "Bug" is functional but wrong

Daily Scrum

• 3 Questions everyone answers:
  • What did I do yesterday?
  • What am I doing today?
  • What issues am I having?
• Each person should answer quickly - 1 or 2 minutes, max
• further discussion happens outside of the Scrum meeting

Productivity of Dev Team

• RAD and other non-review, non-iterative based management causes problems and loss of productivity
• we need constant review of the design to ensure good design
• shorten the feedback loop and get constant review of the design, to always improve the design, via pair programming, work cells, retrospectives, etc.
• good design will cause productivity gains in the development team beyond the capabilities of any tools

Whiteboard Diagramming vs. Details Specs

• White board diagramming and human interaction is always better than detailed documents and specs in UML
• Human interaction leads to knowledge crunching and learning, not just reading a repeating
• Take pictures, don't re-draw in UML; don't waste time with it
• Video the entire conversation is even better, so others can learn from the knowledge crunching that occurs; capture the human interaction, body language, etc.

The topic of Model-View-Presenter starter resources came up at work, today, and I like the list I pulled together. So, I thought I would share with the rest of the world. This is by no means a comprehensive list, or even "the best" list - it's only what I pulled together in 10 minutes.

The original source of MVP :
http://www.martinfowler.com/eaaDev/ModelViewPresenter.html

My own best practices for MVP, following the Passive View mentality
http://www.avocadosoftware.com/csblogs/dredge/archive/2008/02/20/787.aspx

This article should solidify a lot of what I am talking about in my best practices
http://msdn.microsoft.com/en-us/magazine/cc188690.aspx

Jeremy Miller was one of the guys to originally start pushing MVP into the ASP.NET community. He’s got a lot of good articles on his blog
http://codebetter.com/blogs/jeremy.miller/archive/2006/02/16/138382.aspx

This was the article that really solidified my understanding of the basics of MVP, back when I first started learning it
http://www.codeproject.com/KB/architecture/ModelViewPresenter.aspx

Aside from those core MVP articles, there are several key concepts that you’ll want to understand in order to really be able to present / teach MVP, including:

• Single Responsibility Principle
• Open/Closed Principle
• Dependency Inversion/Injection and Inversion of Control
• Liskov Substitution Principle

The key to these principles is really the subtlety that underlines them. For example, the Liskov Substitution Principle isn’t just a fancy name for polymorphism – it’s what makes polymorphism work. You certainly are not required to know these principles forward and backward to be able to implement MVP, but you’ll find that the question of “why?” is easier to answer to if you do know them.

Honestly, when it comes down to it, these principles are what make object oriented development work – they are how you achieve the high cohesion, low coupling, and tight encapsulation that OOD wants. There are plenty of other principles (such as Orthogonality, the law of Demeter, and others) that will help you achieve this as well; but this is a starter-list, not the all encompassing list.

One of the books I've been reading recently, that covers these topics and more, is "Agile Principles, Patterns, and Practices in C#" by Robert Martin and Martin Micah.

I highly recommend this book, not just for these topics. It's a great book, all around.

I'm reading "Lean Software Development: An Agile Toolkit", and the first paragraph under "Tool 11: Queuing Theory" talks about the bottleneck that often occurs in the test lab - not enough testers, too much work for the number of testers, etc.

"We have often heard the lament 'My biggest problem is the testing department.' Now, testing people are very nice people: dedicated, hard working, and very important to the development effort. But there never seems to be enough of them to go around. And although the developers might write their own unit tests, testers frequently do acceptance testing. So, without enough testers, the whole development process bogs down."

The rest of the chapter talks about the queuing theory that can be applied to help alleviate the issue. It's a great chapter with lots of good information. I have a problem with the idea of applying this type of queuing theory to the the test lab, as a bottleneck, though.

First, let me state some assumptions about the primary responsibility of the testers in the test lab:

• Testers are writing automated acceptance tests, for automated regression testing and integration testing
• Testers are also doing human interaction testing, for the "human touch" of usability, etc.

If the testers are the bottleneck, and the two primary functions of the testers are as I have listed, then I think the there is a much more simple solution to the problem, from a lean perspective:

let the developers write the automated acceptance tests.

Assuming that the developers are already writing unit tests, and are therefore capable of writing code to test code, it makes a lot of sense in my mind that the developers should be writing the majority of the automated acceptance tests. It all goes back to the idea of flow - ensuring that the entire system (or process) has a smooth flow from beginning to end. This means that we may need to sub-optimize one area for the benefit of the whole, but the end result is that we will have a better system or process by making the entire flow as smooth as possible.

Counteracting Mura - or "Let's make it smooooooth"

If we are looking at the test lab as a bottleneck - a rough spot in the flow of our software development cycles - then let's take the most simple course of action possible, to reduce that rough spot as much as possible. Rather than spending so much time and effort on queuing theory and implementation, let's find a way to remove the bottleneck.

Assume that the software developers are experts at writing code - and writing code to test their code. Doesn't it make sense, then, that the software developers should be writing the acceptance tests, even if the acceptance tests are being specified by the customer and test lab personnel? If we allow the developers to take a little more responsibility, we may be sub-optimizing the development department a little. But, by doing so we are freeing up the much more scarce resources of the test lab and we can then make adjustments to the test lab's queue and workload, if needed. The idea of leveling out the flow of the system like this can be traced back to the Japanese term, Mura. The Wikipedia entry says it all:

"The fact that there is one operator will force a smoothness across the operations because the workpiece flows with the operator."

In this case, we are calling the combination of production code, unit tests and acceptance tests, the "workpiece". I believe this is a fair assesment, since the code and tests are all going to be based on a feature, use case, or user story. In fact, I would say that the workpeice actually is the feature, use case or user story that is being worked on. The code, unit tests and acceptance tests could be considered the artifacts products by the workpeice flowing through the system. ... but that's just splitting hairs, really.

It's all about Occam's razor, Parsimony, KISS, or whatever you want to call it - the simple solution is often the correct solution (simple, however, doesn't always mean easy).

The Need for the Test Lab

I'm certainly not saying we don't need a test lab. The testers are (supposed to be) experts in interaction testing, usability testing, and adding that "human touch". We absolutely need that perspective on those aspects of software testing that can't reasonably be automated. I am advocating that we find a better way to smooth the flow of the system - rather than apply complex theories and equations to the situation, find a solution that doesn't require anything complex.

Conclusions

In the end, the problem of the test lab bottleneck can be solved many different ways. You might level the system via Pair +1 Programming or some other form of involving the developers in writing the automated acceptance tests. Perhaps you make the testers part of the team and have them writing the automated tests at the same time as the developers writing code. You might still need to employ queuing theory. Either way, try to find the solution that works best to smooth out the process for your team.

Based on the proposed information and syntax from My Previous Post, I've created a basic Design By Contract unit testing framework. The intent of the code so far, is to provide a quick-and-dirty proof of concept. With that in mind, I give you

DBCUnit hosted on GoogleCode

You can get the source code via Subversion:

http://dbcunit.googlecode.com/svn/trunk/ 

Please note that the existing code only supports one assertion so far: Equals. Also, the execution engine is entirely made up of terrible code that assumes a lot of perfect-scenario input. I'm planning to flesh it out more and make the code more sustainable - actually adding Contract specifications for my execution engine, etc. Let me know what you think of the idea and the syntax. Also feel free to join the project and pitch in for syntax and implementation.

The one Contract I have specified so far, just to get rolling, is that a [PreCondition] should be executed once.

[Condition]

public class WhenPreConditionIsPresentInContractCondition

{

    private int preConditionExecutionCount = 0;

    [PreCondition]

    public void PreCondition()

    {

        preConditionExecutionCount += 1;

    }

    [PostCondition]

    public void ThePreConditionIsExecutedOnlyOnce()

    {

        Assert.That(preConditionExecutionCount).Equals(1);

    }

}

To run the test, run build the solution and run the DBCUnit.Console pointing to the DBCUnit.Contracts.dll, like this:

C:\...\> DBCUnit.Console.exe DBCUnit.Contracts.dll

If the test succeeds, there is currently no message printed to the console window. If it fails, it will write out a message saying what value it expected and what the value was. It's all very simplistic at this point, just a proof of concept. I also set up the DBCUnit.Console project to automatically start with the "DBCUnit.Contracts.dll" as the startup parameter, so you can step into the code via debugger and see it in action.

Have fun, and don't laugh too much. This is my first attempt at hacking together a unit testing framework.

There's a lot of talk about Design By Contract (DBC) out there in the development world. Various development languages have varying support for it, but more importantly various processes have various levels of support for it.

It seems, though, that the farther down the path of development we travel, the more important it is for us to consider DBC in the code that we write. Large projects with multiple developers are in great need of DBC. Projects that have publicly distributed API's are in even greater need of DBC. Even if you are working on a simple, one person project for yourself, and you are the only one that will ever use it's methods and objects, I'm willing to bet that you will forget about the assumptions that you are making when writing the methods and objects, at some point.

So where does this distinct need for DBC leave us, in the world of .NET (C#, VB, and the other "common" .NET languages)? We still need a way to enforce DBC, but our language of choice doesn't support it, natively. So we have two real choices (excluding DSL writing, and/or switching languages) - documentation (via code comments or written / published documentation) or Unit Tests.

Yes, that's right - Unit Tests are not just for testing, anymore. Or more correctly, the tests executed by unit testing are not just for the sake of testing. The intention is verify the pre and post conditions of a design-by-contract. Of course, I'm not the first one to suggest this. It's mentioned briefly in "Agile Principles, Patterns, and Practices in C#" by Robert C. Martin and countless other times as well.

I am propose a new unit testing framework. I know, I know... "not ANOTHER xUnit framework... *sigh* ". In this case, I am proposing a semantic change along with the mechanical (syntax) change, specifically for the purpose of introducing unit testing to a group of developers that may not believe in "Unit Testing".

As an example of my proposed syntax, in a file called "SomeContract.cs", this test code would exist:

[Conditional]

public class WhenSomeConditionIsMet

{

    SomeValue someValue;

    [PreCondition]

    public void PreCondition()

    {

        Setup.My.Inputs inputs = Here;

    }

    [Execution]

    public void Execution()

    {

        someValue = Execute.TheContract.With(inputs);

    }

    [PostCondition]

    public void ThenSomeOutputIsSomeValue()

    {

        Assert.That(someValue).Equals("Some Known Value");

    }

    [PostCondition]

    public void ThenSomeOutputIsSomeValue()

    {

        Assert.That(someValue).DoesNotEqual("Some Unknown Value");

    }

}

To make this proposed syntax easier to understand, I'm basing it on the NUnit style of using attributes, at the moment; but it doesn't have to be that way. There is almost a one to one translation between NUnit and this.

1. The [Conditional] attribute is equivalent to [TestFixture]
2. The [PreCondition] attribute is equivalent to [TestFixtureSetup]
3. The [Execution] attribute is equivalent to [Setup]
4. The [PostCondition] attribute is equivalent to [Test]

In this simple example, I don't have an equivalent to [Teardown] or [TestFixtureTearDown]. I'm sure I'll need those at some point, but until I see the need, I'm not going to worry about them. I also don't really care about the Assert syntax. I'm just putting that syntax in place to illustrate the point.

Where I differ from typical NUnit style of testing is that I want to see a single "PreCondition" and "Execution" per test fixture, and have multiple "PostConditions" that only contain the assert statements. This style of test code more closely resembles that of Scott Bellware's SpecUnit.NET and for good reason - I like it. I'm a fan of having as little as possible in the method that does the assert - keep it simple and explicit.

The biggest problem I have with my proposed syntax is a problem inherent to Design By Contract - the idea that you know the object (contract) being executed. A huge part of why I love Behavior/Specification Testing vs. Unit Testing is that Unit Tests and TestFixtures typically tell you that for class/file "XYZ.cs" you have "TestFixtureXYZ.cs". Whereas, Behavior/Specification Testing says that we have "BehaviorSpecification.cs" regardless of the classes used to implement it. I love this about Behavior Driven Development - it freed my mind from the horrible constraints that I saw in standard Unit Testing / TestFixtures. Unfortunately, Design By Contract basically takes us right back to the same place. We are specifying contract (class) "XYZ" so we have a "XYZContract.cs" file to hold all of our [Conditional]s.

...

Does anyone else see any value in this style of unit testing? I can certainly see scenarios where this would appeal to some developers more than the standard xUnit frameworks.

I've been learning a lot lately - reading up on Agile, Lean Manufacturing, Lean Software Development; experimenting with NHibernate, exploring UnitOfWork concepts and generally trying to become a better software developer. In this process of learning, discussion with others, and application of knowledge to my environment, I've found that knowledge is very distinctly different than understanding.

Let's look at my use of NHibernate as an example. 2 years ago, I knew the basics of what NHibernate could do for me - I spent a few weeks learning the very basics to see how it worked and whether or not I wanted to use it. 1 year ago, I actually started using NHibernate for a project and my knowledge of it quickly grew. I knew how to create the appropriate mapping files, configure NHibernate, etc. As this knowledge grew, I became more and more interested in what NHibernate could do and how it could be applied to many different applications and situations. I knew a lot about how NHibernate was built, how it worked, and what it's capabilities were. However, all of this knowledge was not a substitute for a real understanding of NHibernate. My knowledge of how it worked led me to some conclusions that I don't think are correct anymore - trying to apply NHibernate in situations where it really was not the right answer. 

I've seen the same problem occur multiple times in the last 6 months, and in the last 11+ years of my career. There's almost a recognizable pattern to the learning process:

• Learning curve to gain working knowledge
• Knowledge growth and intimate familiarity
• Assumption that knowledge gained is a substitute for experience and understanding
• Attempts to apply knowledge incorrectly / in wrong situation
• Realization that knowledge is not understanding or experience
• Stepping back from knowledge to gain understanding of how/when to apply the knowledge

I've done this with Agile/Lean software development as well, recently. I've gained a significant academic knowledge of agile and it's processes and practices. Some of my knowledge is directly backed up by experience, so I believe I do have some understanding of the agile engineering practices. However, I've let my limited understanding mix itself into my academic knowledge a little too much. I've found myself in situations recently where I'm arguing a logical conclusion to a situation and applying it to another situation incorrectly.

One of my coworkers likes to apply this adage to situations like this:

If a hammer is the only tool you have, everything looks like a nail.

I need to remember to step back from my raw knowledge - be it academic or real world use - and let common sense and experience interweave into understanding or the realization that I don't understand. I need to understand that just because I am currently holding a hammer, and thinking about hammers, that doesn't mean that the problem in front of me is a nail that needs to be hammered.

The realization that knowledge is not a substitute for understanding can be a very distressing and/or humbling experience. The reality of learning, knowledge and understanding, though, is that I can have every last bit of knowledge on a subject - but without experience to guide me, I can't always understand the where/when/why/how of applying that knowledge appropriately.

I'm chugging the cool-aid as quickly as I can mix it. This morning, I opened a pack of index cards with the intention of taking notes in a non-linear fashion (oh, the freedom my brain finally found). By the end of the day, I was hanging user stories on my cubicle wall to manage all of my administrative responsibilities. After all - if it's good for managing software development, why wouldn't it work for managing my daily / weekly administrative tasks (server maintenance, training coordination, permissions on various things, working with various people for various goals, etc).

On the left side of the divide (created by the wall segments - makes it easy) is my back-log. On the right is my in-progress list sorted by highest priority at the bottom. The done list is the trash can you can't see on the floor.

One of the primary goals of Lean Manufacturing in the elimination of waste. This may seem like an obvious goal - after all, who wants waste in their system? What's really interesting, though, is seeing where the term "waste" is actually applied and how Toyota redefined what waste really is.

Muda, the concept of the seven wastes in the Toyota Production System, is a very different concept of waste than most of us would consider. It talks about not only physical waste (trash), but also logical, time, and effort waste. These seven wastes include (from Wikipedia):

• Defects: Quality defects prevent the customers from accepting the product produced.
  
• Overproduction: Overproduction is the production or acquisition of items before they are actually required.
  
• Transportation: Each time a product is moved it stands the risk of being damaged, lost, delayed, etc. as well as being a cost for no added value.
  
• Waiting: Refers to both the time spent by the workers waiting for resources to arrive, the queue for their products to empty as well as the capital sunk in goods and services that are not yet delivered to the customer.
  
• Inventory: Inventory; be it in the form of Raw Materials, Work-In-Progress (WIP), or Finished Goods, represents a capital outlay that has not yet produced an income either by the producer or for the consumer.
  
• Motion: As compared to Transportation, Motion refers to the producer or worker or equipment.
  
• Overprocessing: Using a more expensive or otherwise valuable resource than is needed for the task or adding features that are designed in but unneeded by the customer.

In "Lean Software Development: An Agile Toolkit", Tom and Mary discuss The Seven Wastes and offer a great set of parallels in software development. Rather than copy & paste what they've already said, I wanted to offer my own interpretation of these wastes - some of which are the same as Tom and Mary's.

Defects

This is exactly what it sounds like - a bug, a piece of functionality that doesn't work right, or an exception that gets thrown in the system and causes problems. How many hours, days and weeks have you spent fixing defects in software, in your career? Personally, I can't even begin to accumulate the time spent - it's far too great.

Part of the problem here, and what really makes it wasteful, is when defects are found and fixed in most software development cycles. In Code Complete 2, McConnell talks about the cost of defects in software, relative to when the defect is found and corrected (pages 472 & 473). If the cost of fixing a defect goes up over time, then the amount of waste also goes up. In other words, the higher the cost of fixing a defect, the more waste that defect has generated.

With the increasing cost of defects in mind, we will want to reduce the time to find a defect so that the amount of waste generated can be mitigated. In Lean Manufacturing, this is achieved by only creating what is needed for an actual customer order, right now. If one part is produced with a defect, that one part is scrapped and a new part is created, correcting the defect - the waste is minimal. Imagine a mass-production system where parts are created in batches of ten thousand. What happens if there is a defect caused by the process? We end up with a significant amount of waste, because we will have to scrap the defective parts in batches of ten thousand - the waste here, is significant.

The same principle holds true in software development. If we are writing code for extensive periods of time, and that code is not being tested until the end of that period, then the cost of finding a defect in that code goes up. If we have ten lines of code that rely on the defect working the way it does, it is significantly cheaper to fix than if we have ten thousand lines of code relying on the way the defect works. The easiest way of reducing the number of lines that rely on a defect is by working in small iterations. If that defect goes into test after two weeks, it will be much cheaper to fix than if it had gone in to test after two months. Additionally, if automated tests are done and can be run against the system on an even shorter schedule, then the cost of finding the defect may be even less. If the automated tests are run every twenty or thirty minutes, you may only have two or three lines of code that are affected by the defect.

Overproduction

There are several common terms for overproduction, in software development - Over-Engineering and Coding For The Future, for example. Both of these problems result in the same thing - too many features and more functionality than is actually needed right now. Additionally, overproduction can be called technical debt. The idea is that there is code in our system that will need change - when this code needs to be changed, how difficult is it to change; what is the cost of the change (how much debt are we in).

If a developer is tasked with creating Feature-X, they should focus on the core business value of that feature when writing the code. It's very easy for that developer to think about "what-if" scenarios and "we may need to" future functionality. However, this type of thinking leads directly to overproduction. If the business has not identified the "what-if" scenario, then why should the developer assume that handling the scenario would add business value? If the business has not yet identified Feature-Y or specified any additional functionality in Feature-X, then why should the developer assume that coding for the possibility of Feature-Y is valuable? Take note of the pattern in those questions - assumption. Assumption on the developer's part leads directly to the overproduction of the system being built.

So how do we prevent this overproduction? First and foremost - stop assuming you know what's coming down the line, or what will provide business value. If you believe that the specs you are currently working with are not accounting for an actual problem, then the responsible action is to discuss that issue with the customer / business analysts. Secondly, if you find yourself wanting to code for the feature that may be coming down the line - stop. Don't do it. Even if you know that this feature or functionality is coming down the line, it may not provide any business value for the current feature(s). You may be causing detriments to the business value of what is currently being built. If you truly believe that the feature you want to code is required for the feature that you are coding, then you need to go back to the customer / business analyst again, and discuss it with them. Let the customer tell you whether or not there is value in adding that feature now vs. later (vs. if at all).

Transportation

Think of transportation in the context of knowledge. No one knows everything about a particular domain or project, and possibly an individual feature. We have business analysts and customers precisely because of this. Eric Evans discusses knowledge crunching (the process of learning and gaining an understand of the domain in question, to create a model) in Domain Driven Design, in the first chapter. Tom and Mary talk about the process of learning all throughout the "Amplify Learning" chapter of Lean Software Development. There are countless other books on requirements gathering, model creation and other learning processes for software development, as well.

The problem that we face in our learning is that the transportation of knowledge, from one person to another, will always introduce change. Every person in existence has their own unique way of understanding, based on their experience, existing knowledge and methods of knowledge retention. To illustrate this, look back at your childhood. Did you ever play the "telephone" game where one person whispers in the ear of the person next to them, then they whisper the same thing into the ear of the person next to them, and so-on until the last person in line? What are the typical results of this game? More often than not, the message had some fundamental change to it throughout the transmissions. As simple as the telephone game is, it's a perfect illustration for knowledge transportation that occurs in software development projects.

How many times have your received a functional specification or requirement for a feature, and had no clue what it was trying to convey or what the functionality was supposed to be? This happens on a very regular basis. The problem may not be that the documentation is inadequate, but that the transportation of knowledge from one person's mind onto the paper and into your mind is inadequate - what makes sense to the document writer(s) may not make sense to you, because your contextual knowledge of the subject is different. This problem is exacerbated by design-up-front project methods. The knowledge is often crunched from the customer's mind into the BA's mind, and then trampled down into a linear document or subset of knowledge in a documentation set where the developer has no direct access to the customer - the source of knowledge - for clarification and discernment of the details. Even if the customer is on-site sitting next to the developer throughout the creation of the documentation, there is still a transportation problem in design up front projects. Whether or not the developer understands the information during the conversations, there is little chance that the developer will remember every detail of the information when they finally get down to coding.

The solution for the transportation of knowledge is ensure that the knowledge is transported as little as possible. Rather than focusing the effort of documentation on every last possible detail - creating tomes of documentation that no one is really willing to read - we should focus on creating higher level guidance; enough knowledge to know the direction of the development effort and the features to be created, with the understanding that the final details will come from the customer just-in-time for the development effort. In the world of lean manufacturing, this is typically done through the Kanban system. In software development, we have adopted the same process through the use of User Stories being written on index cards. The stories are not the final word in the development effort - they are merely a placeholder for a conversation between the customer and the developer.

Waiting

This is likely the most obvious of the seven wastes, when applied to software development. Whether you are waiting for a customer or business analyst to be available for questions; waiting for a requirements document to be signed off; or waiting for your next assignment of work, the simple process of waiting creates lost time which directly translates to lost opportunity. In the extreme cases, the problem is compounded by context switching. If you are waiting for a question to be answered - whether this means waiting for a phone call, a document to be written, or just walking down the hallway - you can easily lose your focus or lose you place in the code or coding process. When this happens, most people take a while to restore that focus and get back into the mind set where the new knowledge is useful. In my experience (no real data, just my experience), the average time of context switching within the same project is as little as five to ten minutes for wind down, and at least ten or fifteen minutes for spin-up. If you are context switching between multiple projects, though, expect to spend much longer on the spin-up; hours, even days, depending on the project and time between wind down and spin-up.

If we can ensure that we never wait longer than it takes for us to wind down, then we should be able to eliminate the spin-up period; thus reducing the effort required for reestablishing our focus on the feature(s) at hand. However, the solution for waiting may not be as obvious as the problem. If you are waiting for documentation, or if you are waiting for the customer to sign off on a feature or change, the way to eliminate the waiting is to eliminate the cause. Don't rely on heavy documentation or require customer sign-off on features. If we are working with the customer every hour of every day, eliminating the knowledge transportation issues, then we can also eliminate much of the waiting game by having the customer available. When the customer is available immediately, we have a much greater chance of the conversation beginning within the wind down period. When this occurs, we can stay focused on the problem at hand, and mitigate the spin-up period when we return to the code.

Inventory

Inventory in a manufacturing or supply context is typically a product or component of a product that is in storage and waiting to be used; or "a list of goods and materials held available in stock by a business" (Wikipedia). In software development, we only need to translate "good and materials" into "code and resources":

A list of code and resources held available in stock by a business.

Think of software inventory as unfinished code (as defined in Lean Software Development) or as overproduction of code. If we have unfinished code in our system, this is like having a physical component for a product - the component by itself is not useful until it is in the completed product. Similarly, the unfinished code provides no value to the software until it is finished. Even if the code is complete, if it is not being used in the software then it is dead code and is inventory for the software.

Inventory, in software development, carries a significant technical debt - the unknown factor of whether or not the code actually needs to be in the system. The longer we let inventory live in our code, the more likely we are to forget whether or not that code needs to be there. This inventory also becomes a potential mine field for developers - what happens if a developer sees this dead code and assumes that it is still functional and valid? If we are lucky, the code will work as advertised. The longer the code sits in inventory, though, the more likely it is to be outdated, invalid and eventually non-functioning.

Motion

In addition to physical movement causing waste through waiting (walking down the hall to the BA's office), motion can also be applied to the processes of producing the executable software (not the source code and other resources). How much effort does it take for your team to compile the resources, create the executables, package it into an installation and deliver it to QA, the customer, etc? The goal should be to make this as simple as possible - as little movement as possible.

Are you able to click a button and have the build, package, and delivery processes handled for you automatically? Or does it take a team of developers, build engineers and other persons with a manual process and a checklist of things to do? If your process involves people performing manual tasks - even if that task is simply the order in which multiple buttons get clicked - then you are at risk of creating waste through defects. Every manual motion in your build process is another point at which mistakes can be made, allowing defects to be built or causing the build to fail completely.

Software production (again - speaking about production of executable software, not development of resources or code) is often a difficult, tedious chore, requiring many moving parts - there is no way around this. However, it does not have to be a manual process. Continuous Integration and Continuous Deployment practices are abundant in the world of software development, specifically to address this issue. If you can automate your build process and eliminate the manual movement involved, you can eliminate a huge source of potential waste.

Overprocessing

Overprocessing, or too much process, is abundant in software development. When a customer has a request for a feature, how much process does it take for that feature to become functionality? In design-up-front project life cycles, the process of adding or changing a feature can be very time consuming, involving many steps on both the requester and implementer side:

request; acknowledgement and change management / approval; design and documentation by implementation staff (BA's, developers, etc); sign off of design and documentation by customer; implementation in software; scheduled for testing; tested; and if any issues are found, back to design or implementation for corrections, repeating several steps.

Additionally, how much work does it take for you to get your job done, as a developer? Do you spend as much time updating task / issue management tickets and managing your manager, as you do coding?  Too much process in software development is as bad, if not worse, than no process at all. Unfortunately, the failures of too much process are often seen as an indication of the need for more process, causing further delays and waste in the system.

There are often situations where you cannot do anything about the process that is in place. Life-Critical systems (x-ray machines, air plane guidance systems, etc) are abundant with process for good reason - if anything fails, people could be injured or killed. In the world of business software, though, this is not usually the case. Most business has a level of fault tolerance built into it, and some waste is expected / accepted. This is not a get out of jail free card, though. The more significant the waste, the more repercussions there are - fix the defect; pay for lost revenue; lose your job; etc. If we want to eliminate waste through the simplification of process, then we need to ensure that defects are found and fixed as early as possible, to avoid creating waste for the business using the software.

Process simplification can be done many different ways. Start with the creation of a process flow char, or value-stream chart. Find out how much time and effort is wasted in your processes and look for the waste that can be easily eliminated. Simplifying a single process by removal of waste can lead to a revolution in your process. You may find that the removal of one process negates the need for another process. You may also find that a simple change to one process does the same. If you are able to bring your customer into the work area with your team, and work in short iterations where defects are found and fixed quickly, you will likely see many of your old processes quickly become obsolete.

Conclusions

No one wants waste in their systems. Waste is money lost and revenue not earned. Waste management, though, can be a difficult proposition. Many company are willing to accept waste as the cost of doing business and are not willing to spend money eliminating waste because they do not see the waste or do not believe it can be eliminated. If we are truly going to revolutionize the business we work in, we must eliminate waste. Start where you have direct control or influence. You'll quickly find others wanting to know how you are so productive and soon the entire business will be in full waste reduction cycles.

A while back, I posted a stream of thoughts concerning programming in triples. The basic idea is that we have 2 developers doing pair programming and a third developer doing automated acceptance test development, on the same story at the same time.

Triples Programming Terms

I've been discussing this idea with various people since then, and I'm finding myself referring to this team organization with a specific term, now:

Pair+1 Programming

I don't know if this term has been used previously, or if there is already a term for this. However, I find this to be a good name for the organization that I am talking about.  It really is pair programming, as defined by many common sources, plus one acceptance test developer. The really fun part is that all three of the developers in this unit will switch roles throughout the day. The end result is that we have a very well cross-trained team - each member of each unit will be responsible for writing unit tests, writing production code, writing acceptance tests, and thinking ahead of the current code, throughout any given day.

An Agile Team includes more than just the developers - the customers, the test lead, the technical writers, the project manager, etc., are all part of the team. Pair+1 Programming is not into teams. A "team" implies that they the persons involved are fairly static - they don't move between teams, etc. In order to distinguish the Pair+1 organization from the Team as a whole, I am also going to refer to this group with another name:

Development Unit (Update: based on further discussion, renaming this)

WorkCell

Just as a developer will rotate rolls within the Unit, I believe Pair+1 programmers should move between WorkCells on a regular basis. This will further the cross-training that occurs and help to pollinate the knowledge of one WorkCell into the thoughts and goals of other WorkCells. The migration is likely to be less volatile than the role switching that happens within a WorkCell. For example, developers rotate positions every 2 or 3 hours, within the WorkCell, whereas a developer may only rotate between WorkCells once a day or once a week.

Development Systems

Within each Development Unit, there will be two development systems.

1. The primary system of that Unit will be the pair programming system. This system should have a very large monitor - 30" is preferable - to allow the Pair to easy see everything on the screen.
2. The secondary system will be used for Acceptance Tests and is likely to only need a 22" or 24" monitor, as it will be primarily used by a single developer.

Each of these systems, aside from monitor size, should be a mirror of each other - the same system specs with the same software development packages installed. Every machine used for development should be able to execute any of the tests - unit tests or acceptance tests - no matter the primary use of that development machine. This is required so that the Pair working on the Development machine can execute the Acceptance Tests being created on the Acceptance Test machine; thus ensuring that they are coding appropriately, and passing the expected tests. The Acceptance Test developer should also be able to execute the code being written by the Pair, to ensure that the Acceptance Tests are being written against the API correctly.

Following standard Agile practices, these machines should not be concerned with Email, IM, or other non-development tasks. They should be dedicated entirely to development work. If a person needs Email, IM, etc. then the office space should provide access via shared systems and/or a developer should have their own system (likely a laptop) that they can take with them wherever they go. The point is, though, to remove distraction from the Development Unit. Keep the non-development software off the Unit's systems, as much as possible, to ensure that the Unit can focus on development and not be distracted by 50 different flashing icons and windows on the machine.

Disclaimer

A lot of my thoughts are purely academic, at this point. I am hopefully going to have some actual experience with this environment, soon. As such, I am trying to organize my thoughts so that I can have some direction to start with. I reserve the right to be wrong and change my mind whenever I want, likely re-defining the terms and processes involved in Pair+1 Programming.

In my previous post, I talked about my base repository class that I use, to abstract the NHibernate details away from the actual repository. Now that I have the Do and DoTransaction methods to further the abstraction, I thought it would be good to share my whole abstraction.

using System;

using NHibernate;

using NHibernate.Cfg;

namespace RepositoryBase

{

    public abstract class Repository : IDisposable

    {

        #region Vars

        private static readonly Configuration _config;

        private static readonly ISessionFactory _factory;

        #endregion

        #region Constructor / Destructor

        static BaseDAL()

        {

            _config = new Configuration();

            _config.Configure(typeof(BaseDAL).Assembly, "RepositoryBase.hibernate.cfg.xml");

            _factory = _config.BuildSessionFactory();

        }

        ~BaseDAL()

        {

            Dispose();

        }

        #endregion

        #region Properties

        public ISession Session { get; private set; }

        public ITransaction Transaction { get; private set; }

        #endregion

        #region Methods

        protected void Do(Action unitOfWork)

        {

            try

            {

                OpenSession();

                unitOfWork();

            }

            finally

            {

                CloseSession();

            }

        }

        protected void DoTransaction(Action unitOfWork)

        {

            try

            {

                OpenSession();

                BeginTransaction();

                if (unitOfWork != null)

                    unitOfWork();

                CommitTransaction();

            }

            catch

            {

                RollbackTransaction();

                throw;

            }

            finally

            {

                CloseSession();

            }

        }

        #endregion

        #region Helper Methods

        private void OpenSession()

        {

            if (Session != null) return;

            Session = _factory.OpenSession();

            Session.FlushMode = FlushMode.Auto;

        }

        private void CloseSession()

        {

            if (Session == null) return;

            if (Session.IsOpen)

            {

                Session.Close();

            }

            Session.Dispose();

            Session = null;

        }

        private void BeginTransaction()

        {

            ValidateSession();

            Transaction = Session.BeginTransaction();

        }

        private void CommitTransaction()

        {

            ValidateSession();

            if (Transaction != null)

                Transaction.Commit();

            CloseTransaction();

        }

        private void RollbackTransaction()

        {

            if (Transaction != null)

                Transaction.Rollback();

            CloseTransaction();

        }

        private void ValidateSession()

        {

            if (Session == null)

                throw new ApplicationException("NHibernate Session Not Open.");

        }

        private void CloseTransaction()

        {

            if (Transaction == null) return;

            Transaction.Dispose();

            Transaction = null;

        }

        #endregion

        #region IDisposable Members

        public void Dispose()

        {

            CloseTransaction();

            CloseSession();

        }

        #endregion

    }

}

Note that I am using an embedded resource as my "hibernate.cfg.xml" location. Just change this line to use a file system resource, or whatever you want for your hibernate configuration.

I'm still wanting to re-abstract this into a set of objects that let's me be concerned with transactions and queries at a business level. One step at a time, though. The syntax that I would like to see, at the moment, would be something like this:

public void SomeBusinessValue()

{

    Something something = DoSomething.BusinessRelated();

    SomethingElse somethingElse = SomethingElse();

    Repository.DoTransaction(() =>

    {

        SomeRepository.Save(something);

        SomeOtherRepository.Delete(somethingElse);

    });

}

And if I get really ambitious, I may try to incorporate Castle.Windsor's Automatic Transaction Facility, so that I can have syntax like this:

[Transactional(Transaction.Requires)]

public void SomeBusinessValue()

{

    Something something = DoSomething.BusinessRelated();

    SomethingElse somethingElse = SomethingElse();

    SomeRepository.Save(something);

    SomeOtherRepository.Delete(somethingElse);

}

Of course, the more I travel down this path, the more obvious it is how Rhino.Commons' Repository came along. Heh - I'm only about 2 steps away from completely re-doing Ayende's UnitOfWork.

In fact, I may try to re-use NHibernateQueryGenerator and go for this syntax:

[Transactional(Transaction.Requires)]

public void SomeBusinessValue()

{

    Something something = Repository<Something>.Find(Where.SomeProperty = someValue);

    Something something = DoSomething.BusinessRelated();

    SomethingElse somethingElse = SomethingElse();

    Repository<Something>.Save(something);

    Repository<SomethingElse>.Delete(somethingElse);

}

Wouldn't that be fun... I really do like this syntax. In fact, I spent 9+ months working with it on a project because I love the simplicity of the syntax. In the end, though, I did not understand all of the underlying architecture and abstraction and Ayende set up (he includes the ability to plug in any DAL, including Castle.ActiveRecord, NHibernate, etc) and it caused headaches for me.

...

The major problem I have with this syntax, at the moment, is unit testing the Repository object. I either need to hide these details behind an IWhateverRepository interface (complete with Save, Delete and GetByWhatever methods) like I have been doing, or I I'll need to learn how to unit test NHibernate with in-memory database or something... We'll see where this leads.

(side note: how's that for a "ThoughtStream". )

For the last few months, I've had a very small base class that abstracts out the NHibernate configuration, session creation, etc. It works very well, but is very limited in what it can do. basically, every method in my actual repository implementation would have to open a new session, execute a criteria and close the session. A typical implementation would look like this:

public ICollection<Invoice> GetAll()

{

    ICollection<Invoice> invoices = null;

    try

    {

        OpenSession();

        invoices = Session.CreateCriteria(typeof(Invoice)).List<Invoice>(); 

    }

    finally

    {

        CloseSession();

    }

    return invoices;

}

That certainly is easy and keeps the code fairly clean, removes a lot of duplication, etc.

However, there is a significant limitation - I can't have any code re-use for multi-query scenarios, without duplicating code. In other words, if I want to load that list of invoices and then load some other collection from another repository, I have to use two different Sessions across two different repository implementations. This really becomes an issue when dealing with transactions - I want my entire change set to pass or fail in a single transaction. In my current abstraction, this can't be done.

Fortunately, NHibernate has the solution to my dilemma built right in - all I need to do is create my criteria objects without a session, and then I can execute any / all of them from any session that I want.

public ICollection<Invoice> GetAll()

{

    ICollection<Invoice> invoices = null;

    try

    {

        DetachedCriteria criteria = DetachedCriteria.For<Invoice>();

        OpenSession();

        invoices = criteria.GetExecutableCriteria(Session).List<Invoice>(); 

    }

    finally

    {

        CloseSession();

    }

    return invoices;

}

I don't have a complete abstraction of the separate execution, yet. However, a very basic implementation could look like this (idea stolen from Ray Houston):

protected void Do(Action unitOfWork)

{

    try

    {

        OpenSession();

        unitOfWork();

    }

    finally

    {

        CloseSession();

    }

}

public ICollection<Invoice> GetAll()

{

    ICollection<Invoice> invoices = null;

    DetachedCriteria criteria = DetachedCriteria.For<Invoice>();

    Do(() =>{

        invoices = criteria.GetExecutableCriteria(Session).List<Invoice>();

    });

    return invoices;

}

This simple abstraction provides a lot of benefit for us.

• Eliminates duplicate code (not calling OpenSession / Close Session from all repository methods)
• Removes ugly Try / Catch blocks from Repository methods
• Allows multiple Criteria to be executed from a single Session / Transaction

And most importantly - this gives us the ability to create a better abstraction of NHibernate, to support business level transactions, not just repository level transactions. Obviously, this simple example is not going to provide business level transactions. It does get us down the path, though.

A coworker and I often have conversations about Unit Testing vs. Test Driven Development. Generally speaking, we agree - there are some semantic or mechanical differences in what we're saying, but nothing major and we usually work that out through the conversations, defining what we are saying. Recently he asked if I ever allow myself to write any code without unit tests, or write code before unit testing it. My initial answer was no, not surprisingly. However, after discussing the question and it's implications further, he brought up a good point and a scenario where I highly encourage writing code without tests:

Prototyping (or Spiking, in Agile terms).

I've posted in the past about how I believe that Prototyping A Process is important in software development, so I won't completely re-hash that. Although, the language that I use to describe prototyping may be evolving, the core concepts and process are still in place (the spiking concept is the same as what I called Prototyping).

Here's what ExtremeProgramming.com has to say about Spiking:

"Create spike solutions to figure out answers to tough technical or design problems. A spike solution is a very simple program to explore potential solutions. Build a system which only addresses the problem under examination and ignore all other concerns. Most spikes are not good enough to keep, so expect to throw it away. The goal is reducing the risk of a technical problem or increase the reliability of a user story's estimate.

 

When a technical difficulty threatens to hold up the system's development put a pair of developers on the problem for a week or two and reduce the potential risk. "

This may seem counter to the creed of writing unit tests first and even counter to the creed of not coding for the future. There is a key element in this description, which I believe is not emphasized nearly enough. The code in your spike IS throw-away code. DO NOT copy and paste even one line of code from the spike into the production code.

"Copy and paste is a design error." - David Parnas

When you understand the process, technology or whatever it is that you are learning, well enough, you must step back from that solution and back into your actual project. Then, you continue the test-first process of Test Driven Development - you write your tests for the area that you are covering and then you write the implementation code using the spike as a read-only reference.

So, yes - there is a time and place for writing code without any unit tests; production code is never that place, though.

A lot of people ask these questions when they first start unit testing

• How many unit tests is too many?
• Do I need to cover every property, every individual method, ever object, every ???

The goal of unit testing is to provide 100% test coverage. The reality of unit testing is that you want 95% or more, test coverage. There are occasions when unit testing that one last line of code is horrendously repetitious or you miss something or accidentally couple something too tightly. But wait... there's more... and those seem like lousy excuses that lead to allowing bad design in your code.

Ultra-Fine Granularity is Horrible

If you are writing your unit tests after you write your production code, or if you are writing your unit tests first but are simply going through the mechanical process switch and it doesn't really matter if you write your tests first or not, then the answer is horrible. You'll end up unit testing way more than you need to. For example, I wrote a login screen last year. This login screen has three fields and two buttons on it: Username, Password, a drop list of locations assigned to the username, a Login button and a Cancel button. How many unit tests do you think should be written for this? ... I wrote 27 unit tests to cover every possible edge case in the presenter that controlled this view. What a giant horrible mess - changing anything in that login screen was almost as bad as not having it unit tested at all (well ok... nothing is that bad)

I ended up unit testing setting an individual property, and then checking to make sure that property was stored correctly. I unit tested individual method calls with only the username set, or only the password set, or only the location set, or only whatever combination of those set. I unit tested loading the list of locations for the username, and ensuring that the location selected is valid for the user. I unit tested what would happen is an invalid location was selected or a null location was selected... every possible edge case was unit tested and it drove bad design into the application because no one wanted to go through the pain of having to change all of those unit tests at that level of granularity.

Step Up To The API

Just unit testing your code is a great way to ensure that you are writing way more unit tests than you need. Chances are, the code you are writing is not very cohesive and you will end up unit testing the read and write of individual properties rather than just unit testing the business value (process) that actually reads / writes the individual properties. That is to say, your unit tests should be written at one or two steps above ultra-fine granularity. Don't test the individual properties, test that API that you want to call, that has business value.

So, how do you account for 100% code coverage if you are not unit testing the properties and all of the edge cases?

Never write code that you don't need, right now. If you are writing a unit test and the test or the implementation needs a property, then you create that property for that unit test at that time. This does not mean that you write a bunch of get / set property unit tests, just so you can unit test the properties. This means that you specify the business value API in your unit test, and by virtue of having business value, you will likely have various properties associated with the classes in that API. The same is true for edge cases - if the business value of the unit test does not handle the edge cases, then there are no edge cases. Only when you have business value specifying an edge case, do you need to write a unit test for the edge case and possibly modify code to handle the edge case.

Ok, then what happens if your code changes and you don't call that property in the original unit test, anymore?

Never leave dead code in your system. Ever. Period. End of discussion. If you change your unit tests because the design of the object(s) change, and you are no longer using a property - delete the property! If you delete it and you find that you can't compile the code any longer because other parts of the system need that property, then you need to evaluate whether or not that property is really providing value to those other places vs. changing those other places to match the new design.

Test First vs. Test After

A big part of figuring out how many unit tests you need is understanding the functionality of the system. You should be writing a unit test for every functional point of the code, achieving 100% code coverage. The problem with the original question of how many unit tests to write, though, is that there is a hidden assumption in that question:

"I wrote my code, now how many tests do I need, to cover it correctly?"

This question is an underlying problem in Unit Testing and simple Test First development. If you are just unit testing your existing code or only going through the mechanical process switch of writing a unit test first, but not really using the test to drive your design, then you are likely not going to see some of the major benefits of Test Driven DESIGN / Development: not writing code you don't need, and creating the API that you want to call instead of the API coming together haphazardly as a bi-product of writing code first.

When you take the step up to unit testing the API, it becomes more apparent that you really want to specify the API before you write it. If you specify the API before you write it, then you are one step closer to true Test Driven Development. Don't expect the test to design your code for you. Use the test to flesh out your design before you write any code.

Test Driven DESIGN / Development

Would you rather:

Write 50+ lines of code into your model, then write a unit test that shows an ugly API causing you to go back to the code and re-write it in the hopes that it will produce a better API, most likely repeating this process once or twice until you get frustrated with changing your code because it takes so long

or

Write 5 lines of unit test code, specifying the API that you want, realizing that it's not going to work and changing 2 or lines of that test, going through this cycle 5 or 6 times until you have the API that you really do want to call; then implementing the API in the 50+ lines of code and being done with it

I'll take #2. I don't like rewriting large chunks of code. Rewriting 2 or 3 lines of code is easy - I'll do that any minute of any day. Chances are, if you are willing to write the correct number of unit tests by specifying the higher level API in your unit tests, you will gravitate toward designing your API in your unit tests.

TDD Misconception:

TDD is NOT a design tool. It is not "the answer". Is will not design your application for you. It will not solve your problems for you. If you don't know how to design software, then you need to get some training on design patterns, loose coupling through single responsibility and separation of concerns, and various other core foundations of good Object Oriented Development.

In reality, Test Driven Development is just an easier way of saying this:

"Design your API in the context of a unit test, so that you have your API implementation covered by unit tests before you even write the implementation."

Conclusions:

In the end, we can answer the original questions from this post by re-stating Test Driven Development as a software development guideline:

"Design via code, unit testing 100% as you go."

As a kid, I was never part of the boy-scouts or anything; but my family and I went camping a lot, and I went camping with my youth group on several occasions. I remember hearing my parents and the various youth leaders talking about we should always leave the camp site cleaner than we found it. I always thought this was annoying - why should I clean up someone else's mess? If I clean up my own mess, isn't that good enough?

Yesterday, while helping a coworker fix some bugs in an application that I wrote around a year ago, I was suggesting ways to improve various parts of the code; move this property to a parameter of that method, make this method private and only call it from here in the owning class, and items as simple as making an if-then statement easier to read.

After a few of these suggestions, he asked me if I always clean up the code that I'm working with, even if the bug is not directly related to the code that we are cleaning up. My answer was emphatically, "yes". If I'm reading code, trying to find a bug and I'm having a hard time understand what's going on with the code, then it becomes much more difficult to find the actual bug. Even if this code does not end up being part of the bug I was looking for, by cleaning up the code I am making it more likely that I will be able to understand what this code is doing the next time I have to look at it.

Here's my basic perspective that drives all of this: if you have a hard time reading the code and seeing what it is doing, chances are, you or someone you know will have to debug that code at some point. I don't want to debug hard to read code - that's annoying, at best. I want to debug code that is easy to read and easy to understand. And I certainly don't want to make any of my coworkers debug hard to read code. I try not to torture coworkers like that. So, if my motivation is to not debug hard to read code, then doesn't it make sense that I would want to clean up that ugly code? It makes sense to me...

What does this really come down to, then? Two things:

1. Leave the code cleaner than when you arrived, by
2. Micro-refactoring - make that one line of code easier to read

(By the way - there's no such thing as "micro-refactoring". Refactoring, by definition, is exactly what I described above. Stop trying to change the architecture and learn to change that one line of ugly code. By doing this, you'll find that the architecture does change, because you clean up more than you realize and change become natural.)

I just can't help myself... I had to flesh out the Account Transfer behavior from my previous post. I was interested in two things:

1. Adding a second set of Acceptance Criteria for a valid transfer
2. Implementing the crazy looking "account.Transfer(ammount).To(account)" syntax

Expanded Acceptance Criteria

Existing User Story:

As an account holder,
I want to transfer money between accounts
So that I can avoid overdraft fees

Acceptance Criteria:

1. Given a transfer between accounts,
   When the requested transfer amount is greater than the originating account's balance
   Then the transfer should fail and the balance of the originating and receiving accounts should not change

 

2. Given a transfer between accounts,
   When the requested transfer amount is less than the originating account's balance
   Then the transfer is successful, the originating account is debited the transfer amount and the receiving amount is credited the transfer amount

Implementing The Transfer.To Syntax

This was the fun part, really - seeing if I could implement the API that I wanted to see. It turned out to be significantly easier than I had thought it would be.

I started with the Account object, and added the "Transfer" method. I knew that this method needed to return an object so that I could have the "To" method accept an Account object. A few moments of thinking this through and I decided to go with an object called TransferCriteria. This would let me collect all of the information that I need, about the transfer.

public TransferCriteria Transfer(double ammount)

{

    TransferCriteria criteria = new TransferCriteria(this, ammount);

    return criteria;

}

The end-goal of this syntax model is to return an AccountTransfer object, directly. So, the TransferCriteria.To method then creates an AccountTransfer object, using the TransferCriteria.

public AccountTransfer To(Account receivingAccount)

{

    ReceivingAccount = receivingAccount;

    return new AccountTransfer(this);

}

In the end, I have an AccountTransfer object that contains all of the information I need - the originating account, the receiving account, and the transfer amount. With that in place, the rest of the code is simple.

Behavior Specification

AccountTransferSpecifications.cs

using AccountSample.Domain;

using NUnit.Framework;

using SpecUnit;

namespace AccountSample.Specifications

{

    [TestFixture]

    [Concern("Account Transfers")]

    public class When_The_Requested_Transfer_Amount_Is_Greater_Than_The_Originating_Account_Balance : ContextSpecification

    {

        #region Context

        private const double originatingAccountBalance = 100;

        private const double receivingAccountBalance = 0;

        private const double requestedTransferAmmount = 125.95;

        private Account originatingAccount;

        private Account receivingAccount;

        private AccountTransfer accountTransfer;

        protected override void Context()

        {

            originatingAccount = new Account(originatingAccountBalance);

            receivingAccount = new Account(receivingAccountBalance);

            accountTransfer = originatingAccount.Transfer(requestedTransferAmmount).To(receivingAccount);

        }

        #endregion

        #region Observations

        [Test]

        [Observation]

        public void The_Transfer_Should_Fail()

        {

            accountTransfer.Status.ShouldEqual(TransferStatus.Failed);

        }

        [Test]

        [Observation]

        public void The_Originating_Account_Balance_Should_Not_Be_Changed()

        {

            originatingAccount.Balance.ShouldEqual(originatingAccountBalance);

        }

        [Test]

        [Observation]

        public void The_Receiving_Account_Balance_Should_Not_Be_Changed()

        {

            receivingAccount.Balance.ShouldEqual(receivingAccountBalance);

        }

        #endregion

    }

    [TestFixture]

    [Concern("Account Transfers")]

    public class When_The_Requested_Transfer_Amount_is_Less_Than_The_Originating_Account_Balance: ContextSpecification

    {

        #region Context

        private const double originatingAccountBalance = 100;

        private const double receivingAccountBalance = 0;

        private const double requestedTransferAmmount = 25;

        private Account originatingAccount;

        private Account receivingAccount;

        private AccountTransfer accountTransfer;

        protected override void Context()

        {

            originatingAccount = new Account(originatingAccountBalance);

            receivingAccount = new Account(receivingAccountBalance);

            accountTransfer = originatingAccount.Transfer(requestedTransferAmmount).To(receivingAccount);

        }

        #endregion

        #region Observations

        [Test]

        [Observation]

        public void The_Transfer_Is_Successful()

        {

            accountTransfer.Status.ShouldEqual(TransferStatus.Success);

        }

        [Test]

        [Observation]

        public void The_Originating_Account_Is_Debited_The_Transfer_Amount()

        {

            const double newBalance = originatingAccountBalance - requestedTransferAmmount;

            originatingAccount.Balance.ShouldEqual(newBalance);

        }

        [Test]

        [Observation]

        public void The_Receiving_Account_Is_Credited_The_Transfer_Amount()

        {

            const double newBalance = receivingAccountBalance + requestedTransferAmmount;

            receivingAccount.Balance.ShouldEqual(newBalance);

        }

        #endregion

    }

}

Behavior Implementation

Account.cs

namespace AccountSample.Domain

{

    public class Account

    {

        #region Properties

        public double Balance { get; private set; }

        #endregion

        #region Constructor

        public Account(double currentAccountBalance)

        {

            Balance = currentAccountBalance;

        }

        #endregion

        #region Methods

        public TransferCriteria Transfer(double ammount)

        {

            TransferCriteria criteria = new TransferCriteria(this, ammount);

            return criteria;

        }

        public void Debit(double ammount)

        {

            Balance -= ammount;

        }

        public void Credit(double ammount)

        {

            Balance += ammount;

        }

        #endregion

    }

}

AccountTransfer.cs

namespace AccountSample.Domain

{

    public class AccountTransfer

    {

        #region Vars

        private TransferStatus _status = TransferStatus.None;

        #endregion

        #region Properties

        public TransferStatus Status

        {

            get { return _status; }

            private set { _status = value; }

        }

        public double TransferAmmount { get; private set; }

        public Account OriginatingAccount { get; private set; }

        public Account ReceivingAccount { get; private set; }

        #endregion

        #region Constructor

        public AccountTransfer(TransferCriteria criteria)

        {

            OriginatingAccount = criteria.OriginatingAccount;

            ReceivingAccount = criteria.ReceivingAccount;

            TransferAmmount = criteria.TransferAmount; 

            Execute();

        }

        #endregion

        #region Methods

        private void Execute()

        {

            if (TransferAmmount > OriginatingAccount.Balance)

            {

                Status = TransferStatus.Failed;

            }

            else

            {

                OriginatingAccount.Debit(TransferAmmount);

                ReceivingAccount.Credit(TransferAmmount);

                Status = TransferStatus.Success;

            }

        }

        #endregion

    }

}

TransferCriteria.cs

namespace AccountSample.Domain

{

    public class TransferCriteria

    {

        #region Properties

        public double TransferAmount { get; private set; }

        public Account OriginatingAccount { get; private set; }

        public Account ReceivingAccount { get; private set; }

        #endregion

        #region Constructor

        public TransferCriteria(Account originatingAccount, double transferAmount)

        {

            OriginatingAccount = originatingAccount;

            TransferAmount = transferAmount;

        }

        #endregion

        #region Methods

        public AccountTransfer To(Account receivingAccount)

        {

            ReceivingAccount = receivingAccount;

            return new AccountTransfer(this);

        }

        #endregion

    }

}

And finally, TransferStatus.cs

namespace AccountSample.Domain

{

    public enum TransferStatus

    {

        None = 0,

        Failed = -1,

        Success = 1

    }

}

Conclusion

I am madly in love with Behavior Driven Development.

In A Previous Post, I talked about how I'm jumping on board the BDD wagon, and I gave an example of a TDD style Unit Test that was close to BDD style, based on how I worded the test. I've decided to test that theory and follow the progression from a standard Unit Test as I was writing them, into a BDD test system such as SpecUnit.NET.

Typical Requirement, Typical Test

Let's look at a very basic requirement, derived and stolen from the presentation done by Scott Bellware at the Agile Austin User Group meeting that I attended last week.

If an account holder has multiple accounts and one is low on funds, allow the account holder to transfer funds between accounts. If the amount to transfer is greater than the amount in the originating account, then the transfer should not be allowed and the balance of both accounts should remain the same. This will let the account holder avoid overdraft fees on the account with insufficient funds.

A basic TDD implementation may look like this:

[TestFixture]

public class AccountTransferTestFixture

{

    [Test]

    public void TransferFailsIfRequestedAmmountIsGreaterThanOriginatingAccountsBalance()

    {

        double fromAccountStartingBalance = 100;

        double toAccountStartingBalance = 0;

        double requestedTransferAmmount = 125.95;

        Account originatingAccount = new Account(fromAccountStartingBalance);

        Account receivingAccount = new Account(toAccountStartingBalance);

        TransferStatus transferStatus = AccountTransfer.Transfer(originatingAccount, receivingAccount, requestedTransferAmmount);

        Assert.AreEqual(transferStatus.Failed, transferStatus);

        Assert.AreEqual(fromAccountStartingBalance, originatingAccount.Balance);

        Assert.AreEqual(toAccountStartingBalance, receivingAccount.Balance);

    }

}

There is nothing particularly wrong with this unit test. It looks like decent code - appears to be well organized, makes good use of separated objects, and checks to see that everything is as it should be.

Change the Test

Now let's change up the test a bit and move it towards BDD style, the way I have been writing my unit tests recently. The first thing we'll need to do is change the name of the test. Secondly, let's rewrite the object model to be a bit more clear on what's really going on with the transfer.

[TestFixture]

public class AccountTransferTestFixture

{

    [Test]

    public void WhenARequestedTransferAmmountIsGreaterThanTheOriginatingAccountBalance_ThenTheTransferShouldFailAndTheAccountBalancesShouldNotBeChanged()

    {

        double originatingAccountBalance = 100;

        double receivingAccountBalance = 0;

        double requestedTransferAmmount = 125.95;

        Account originatingAccount = new Account(originatingAccountBalance);

        Account receivingAccount = new Account(receivingAccountBalance);

        AccountTransfer accountTransfer = originatingAccount.Transfer(requestedTransferAmmount).To(receivingAccount);

        Assert.AreEqual(TransferStatus.Failed, accountTransfer.Status);

        Assert.AreEqual(originatingAccountBalance, originatingAcount.Balance);

        Assert.AreEqual(receivingAccountBalance, receivingAccount.Balance);

    }

}

The first thing you'll notice is that the Unit Test's name is significantly closer to the business requirement. This is desired, because it allows a develop and a non-developer to both see what this unit test is really supposed to be doing - how the system should behave when something happens. Secondly, notice the differences in the AccountTransfer object. In the first UnitTest, AccountTransfer was used to attempt the actual transfer and return the status of that transfer. In this test, though, AccountTransfer is an object that is produced by the Account.Transfer method. And lastly, notice the change in APIs to do the actual transfer. Remember that in Test Driven Development, you should be designing the API that you want to call - and then you implement it. I want my API to be very expressive - almost english. This particular change is really a matter of preference, though.

The one thing that has not changed is the list of Asserts. We are still checkign to make sure that the transfer status is "Failed" and that the balance of the originating and receiving accounts is unchanged.

Change the Requirement's Format

Now let's take this example into BDD land.

The first thing we need to do is make sure we have enough information to actually test the behavior of the system. This involves creating a User Story from the requirement that we were given. I won't go into depth on User Stories, here. Just keep the basic format of a User Story in mind, borrowed from Dan North:

As a [role], I want [Feature], so that [benefit]

Let's apply this to the requirement, above:

As an account holder,
I want to transfer money between accounts
So that I can avoid overdraft fees

We can't work on user stores alone, though. The only way a User Story can have any value to a software developer, is when we add Acceptance Criteria to the story. Again, I won't go into detail. Here is the basic format that Acceptance Criteria should follow, borrowed from Dan North, again:

Given [context] [and [some more context], …], When [event], Then [outcome] [and [another outcome], …]

Now let's apply the acceptance criteria to our story

Given a transfer between accounts,
When the requested transfer amount is greater than the originating account's balance
Then the transfer should fail and the balance of the originating and receiving accounts should not change

Notice what we have done with the original requirements, compared to the User Story and Acceptance Criteria. Essentially, we have broken the requirement out into two separate parts that provide more value to the business and to the developer. The User Story is the real business value - what do you want to do, and why. The Acceptance Criteria is where the Developer receives the knowledge of how the system should actually work. Both of these are english statements, though. Nowhere in the Story or Criteria, have we specified any language that deals with a computer, software, or any other technical or implementation detail.

Unit Test the Behavior

With the User Story and Acceptance Criteria separated now, we can write the software and test it. Rather than doing standard Test Driven Development and writing a Unit Test, though; let's focus on Behavior Driven Development and test the behavior of this software using the specified behavior.

This example code is still using NUnit as the test framework, to illustrate that point that you don't need a specific BDD test framework. You can fit BDD style tests into a standard xUnit framework. However, you might feel that it is a little mashed-in and not quite the fit that you want.

namespace Given.A.Transfer.Between.Accounts

{

    [TestFixture]

    public class When_The_Requested_Transfer_Amount_is_Greater_Than_The_Originating_Accounts_Balance

    {

        double originatingAccountBalance = 100;

        double receivingAccountBalance = 0;

        double requestedTransferAmmount = 125.95;

        private Account originatingAccount;

        private Account receivingAccount;

        private AccountTransfer accountTransfer;

        [SetUp]

        public void ContextSpecification()

        {

            originatingAccount = new Account(originatingAccountBalance);

            receivingAccount = new Account(receivingAccountBalance);

            accountTransfer = originatingAccount.Transfer(requestedTransferAmmount).To(receivingAccount);

        }

        [Test]

        public void The_Transfer_Should_Fail()

        {

            Assert.AreEqual(TransferStatus.Failed, accountTransfer.Status);

        }

        [Test]

        public void The_Originating_Account_Balance_Should_Not_Be_Changed()

        {

            Assert.AreEqual(originatingAccountBalance, originatingAccount.Balance);

        }

        [Test]

        public void The_Receiving_Account_Balance_Should_Not_Be_Changed()

        {

            Assert.AreEqual(receivingAccountBalance, receivingAccount.Balance);

        }

    }

}

Notice what we have done with the Acceptance Critiera - we've directly turned it into the tests. We have specified how the system will behave with the various part of the criteria being mapped directly to the various parts of the tests. The [context] has been turned into a namespace; the [event] has been turned into the text fixture's class name; and each of the [outcome] has become a test.

But, why bother splitting all of the parts of the behavior specification out like this? Why not leave it as the unit test was in the previous example? The answer is really the business value of the test and the readability / grok-ability of the tests for developers. I can show anyone on the team - a business analyst, the customer, or a developer - the overview of this specification, and it should be easily understood. A business analyst or customer may not understand the technical details of the Assert statement, or the reason that we have variables declared. However, if they see the collapsed code, it becomes very obvious that the developer has implemented the requested Acceptance Criteria for the user story.

Specify the Behavior

Now that we've seen how to stuff a behavior's specification into an xUnit testing framework, let's look at how to implement it in a testing framework that is designed for BDD: SpecUnit.NET.

namespace AccountTransferSpecifications

{

    [Concern("Account Transfers")]

    public class When_The_Requested_Transfer_Amount_is_Greater_Than_The_Originating_Accounts_Balance: ContextSpecification

    {

        double originatingAccountBalance = 100;

        double receivingAccountBalance = 0;

        double requestedTransferAmmount = 125.95;

        private Account originatingAccount;

        private Account receivingAccount;

        private AccountTransfer accountTransfer;

        protected override void Context()

        {

            originatingAccount = new Account(originatingAccountBalance);

            receivingAccount = new Account(receivingAccountBalance);

            accountTransfer = originatingAccount.Transfer(requestedTransferAmmount).To(receivingAccount);

        }

        [Observation]

        public void The_Transfer_Should_Fail()

        {

            accountTransfer.Status.ShouldEqual(TransferStatus.Failed);

        }

        [Observation]

        public void The_Originating_Account_Balance_Should_Not_Be_Changed()

        {

            originatingAccount.Balance.ShouldEqual(originatingAccountBalance);

        }

        [Observation]

        public void The_Receiving_Account_Balance_Should_Not_Be_Changed()

        {

            receivingAccount.Balance.ShouldEqual(receivingAccountBalance);

        }

    }

}

The major differences between an xUnit framework as a SpecUnit framework, is that the idea of Context is a first class citizen. Aside from that and a few changes to the attribute names used, a SpecUnit test class looks surprisingly similar to the behavior test in NUnit. You'll also notice that we're not using Assert.Equals() anymore. Rather, we're taking advantage of .NET 3.5's Extension Methods and providing a expressive statement. This is somewhat of a syntax preference, though.

We can still compress this code and show it to a business analyst or customer. Even better - SpecUnit.NET comes with a simple report generator that produces an HTML report document based on the specification tests, themselves. The output is essentially a cleaned up version of the collapsed code in an HTML format that anyone can read.

Conclusions: xUnit vs. SpecUnit

The real difference between an xUnit Unit Test and a SpecUnit Specification Test is semantics and a few mechanics. Whether you are using an xUnit framework or a SpecUnit framework, Specification Testing and Behavior Driven Development have a significant benefit to the business when compared to Unit Testing and Test Driven Development alone. As I've said before, this does not mean that TDD is dead or irrelevant. This is only to say that TDD has evolved - it has grown an attachment to Domain Driven Design and Scrum project management, giving it a new home and a new life on it's own. Consider the following:

A Unit Test has the connotation of testing an individual unit of code - a System Under Test as some have called it.

When you pass the value "123" into object XYZ's, ABC method, the result is a change to Foo's Bar property and Widget's Digitize method was called.

Specification Testing, and Behavior Driven Development, on the other hand are all about behavior.

In the context of XYZ, when ABC is done, then the Foo's Bar has been Digitized.

Which would you rather show a business analyst, or the customer? Chances are you could explain the details of the first so that they would understand it... and chances are, that they would understand the second without any need for explanation.

(reposting a few items from my old blog to my new one)

...

I hereby declare my love affair with Test Driven Development over, and my new infatuation with Behavior Driven Development beginning. Call me a cool-aid drinker, a bandwagon jumper, or whatever you want... I finally get it...

When I write TDD / Unit Test code like this:

[Test]
public void WhenMagic8BallIsShaken_ThenAnAnswerIsGiven()
{
  //insert code and asserts, here.
}

(and yes, this is an actual example of a unit test I wrote last week, during a TDD training session I was doing for some coworkers), what am I really doing? I'm not just doing TDD, I'm not just Unit Testing... I'm specifying the behavior of the system; I'm specifying the experience of using the magic 8 ball; and really, I'm only about one step away from the mechanics of Behavior Driven Development.

It's an eye-opening realization for me, and gets me all kinds of excited and re-invigorated to learn more.

Now, I'm not going to say that TDD is dead or invalid - it's more like saying I finally see how Domain Driven Design and Test Driven Development are truly married together... and then you change the semantics of the name to fit that marriage.

It's also worth noting that this jump, for me, is purely from the engineering perspective so far. I still have no real experience with a true agile development project / process. Although I'm slowly stepping closer to that - the more I see in the engineering practices, the more I see the need for the project management practices.

... more to come, as I travel down this path.

(reposting a few items from my old blog to my new one)

...

Another interesting side effect of my conversations with Scott Bellware, today... I'm not sure if he agrees with these conclusions, but I certainly didn't come to them by my own thoughts alone.

My understanding of the Test Lab or QA department in a software development shop has been based on the idea that the QA team members are not developers - and that they shouldn't be, because a developer will write tests that they know will pass. Well, as it turns out - the reason was correct, but the conclusion was wrong.

In reality, a software development project needs to automate every level of testing - all the way out to the point where the actual customer is sitting down and using the software. This includes: Unit Testing, Integration Testing, and UI Testing. Whether or not you distinguish between these three levels is irrelevant; the testing of the system must be automated as much as possible. This leads us to the next question: What is an automated test? Typically speaking, it's code. Who writes code? Software developers. Do you see where this is heading? maybe?

If software is developed by software developers (duh), why is the software that tests the software not written by software developers?

Lets take this to the agile world of 2 week iterations (or sprints or whatever you want to call them). If the software requirements are being defined every two weeks, then the tests are being defined every two weeks as well. If we are automating the tests during these two week cycles, and the tests are written as code, then why are the software developers not writing the tests? The obvious answer is my reason above - the developer will write the test that they know will pass, because they wrote the code that is being tested. So don't let that happen - don't let the developer who is writing the code, write the integration / UI test code. Then, how do we let a developer write the integration / UI test code? Triples Programming.

The standard XP / Agile practice currently has developers working in pairs - one driving the keyboard / mouse, and one thinking two steps ahead. I say we add a third role and a third person to the mix - one developer to write the integration / UI tests at the same time that the unit tests / production code are being written. What you CANNOT do, though, is include that third role in only two people. The moment you break down to two people, you need to remove the integration / UI test writing or you end up with the original dilemma. You also need to ensure that the developers in the triple, are swapping positions on a regular basis - make sure everyone gets a change to drive the keyboard, think ahead, and code the integration tests.

By programming in triples, you get the advantage of specifying the integration tests as soon as the code is functional. You don't lose motivation to write those test, you don't lose clarity of what the functionality is supposed to be, and you don't have to re-discover the same functionality and clarity at a later time with a new set of minds (the QA team).

So, what happens to the QA team? They are still needed, certainly. But now they can focus on their subject matter expertise - the human interaction of manual and random of software testing by using the actual application. i.e. the parts of testing that can't (or shouldn't) be automated.

...

Ok, ok, ok... if you really want to stick with pair programming you can take this concept and assign 2 teams per story; 1 team to unit test / write production code; and 1 team to integration / UI test.

(reposting a few items from my previous blog to my new one)

...

I follow a lot of Agile Engineering Practices, at this point in my career: Test Driven Development, Domain Driven Design, Single Responsibility Principal, Separation of Concerns, Inversion of Control, Dependency Injection, etc. To get to this point, I've basically taught myself - I read blogs, books, example code, and work my butt off to make sure I know what I'm talking about so that I can stay a few steps ahead of my team(s). In all this effort, learning, failure, success and otherwise, I've learned a few key things about software development as it relates to the Agile world.

1. Unit Testing is an absolute must, even if you don't do Agile
2. Unit Testing is not Test Driven Development
3. Test Driven Development is Agile Engineering, not Agile Process
4. Agile Engineering alone, does not make Agile software development

1. Unit Testing is an Absolute Must

Whether or not you are trying to do any sort of agile software development, or even test driven development; if you do not unit test your code, you are opening your code up for breaking unexpectedly. In the worst case scenario, you are preventing your code base from being able to change with the needs of the business and causing crippling problems by not keeping your code separated appropriately.

Think about the goals of object oriented development: well encapsulated, highly cohesive and low coupled objects that can easily be re-used. These goals are very high, very lofty and they may be pie-in-the-sky dreams for some developers. So how do you achieve such goals? Single Responsibility Principle, Separation of Concerns, and Inversion of Control. All of these practices are much closer to the implementation level of code - they tell us to keep our code as small as possible; only write the code that needs to be written, in this class. When you see code that does not quite fit into the class you are currently writing, you look for a better place to put that code. This means that you need to split data access out of your business processing, user interface out of your business processing, and often times you will need to split your business process out into multiple classes.

The hard part is, single responsibility, separation of concerns, etc. are not one-time coding practices per class. They are processes that require a continuous series of design, redesign, and refactoring of your code. Don't assume that an object you write, now, will be the same 10 minutes or 10 days from now. If you find yourself in a situation where two objects are very similar, you may have an opportunity to create a better abstraction that can fit both scenarios - whether it becomes a single object, a template pattern, strategy pattern, or other solution.

If you are continuously redesigning your code to fit the business needs, and you are continuously separating your code into more and more classes, creating the high cohesion and loose coupling of true OOD; how do you manage the process and ensure that you are not breaking your application as you go? The answer is simple: Unit Testing. If you are not unit testing your application, then you do not know if the application works until you actually run it and try to use it. If your application is small - one or two classes, one or two screens - this may be reasonable and you may just run the application in it's entirety to see if it works. If your application is medium to large, though - one or two hundred or thousand classes and screens - you will quickly find yourself in a situation where you cannot expect to run the entire application in a reasonable amount of time and you are likely to miss some of the features and functionality if you do try to run it all. The only reasonable way to ensure that your code still works, is to have an automated test suite that can be executed on a regular basis. The easiest way to achieve this, at a very low level, is to do Unit Testing and ensure that your code is 100% covered with the unit tests. By unit testing every line of code in your system, you will have a rapid feedback tool that tells you when you break something. This rapid feedback will allow you to quickly fix anything that is broken, compared to the "run it and see" process.

With all that being said, it's very important to understand ...

2. Unit Testing is not Test Driven Development

Test Driven Development involves Unit Testing. In fact, Test Driven Development doesn't exist without Unit Tests. However, just because you are Unit Testing, that does not mean that you are doing Test Driven Development. When it comes down to it, Unit Testing is all about insurance - make sure your code is covered because change will happen, and change breaks things. Test Driven Development, on the other hand, is all about software design; writing your unit tests first, to drive the design of the software that you are really writing. TDD is not just Unit Testing; TDD is not just Test-First-Development, either. True TDD is the practice of designing the APIs in your code, by writing out the API that you want to see before you try and implement it.

To do any sort of software development, you need to have some basic functional requirements - what is this software going to do, and why? When you start breaking these requirements down into pieces that are small enough to implement, you can begin to see how the software will look - what classes will be there, what processes will run, and how it all fits together.

When you have good requirements it's easy for a developer to see some of the low level code implementation - using whatever design pattern to create these classes, with this code underneath so that this feature can call into them and do something... TDD, though, wants you to step back a few paces. Don't worry about the low level implementation details, yet. Let's focus on how you want to call into the implementation - what is the API that you want to call, in order to do these processes. Rather than looking at a requirement and saying that you don't know what the API will look like because they have not implemented any of the details, TDD forces you to think backwards - you don't know what the implementation is, but here is the API that you want to call.

3. TDD is Agile Engineering Only

There is a huge disconnect from Test Driven Development, to a true Agile software process. Agile Engineering is a great set of practices that every developer should adopt. This includes Test Driven Development, Domain Driven Design, even Pair Programming. However, these engineering practices can and do fit into any software development lifecycle: RUP, Waterfall, iterative waterfall, and everything else you can image, can all have Agile Engineering practices put in place. I would go so far as to say that we shouldn't even call it Agile Engineering - rather, we should just call these good software engineering practices.

To illustrate this - look at the projects that I work on. They are all standard waterfall type projects. We've implemented these engineering practices in every project that I have lead, for the last 2 years, though.

4. Agile Engineering is not Agile Software Development

Real Agile software development is more than just good engineering practices. It encompasses the entire software development lifecycle - from requirements gathering to requirements definition, continuous discussions with the customer to elaborate on those requirements, discussions with the customer to determine what requirements go into what release and iteration, and more discussions with the client to ensure that the software is heading down the path that they actually want. Have you noticed a trend, here? There's a lot of constant, continuous discussion with the customer in real Agile development. This is an absolute must, in Agile work. If you don't stay in regular communications - as in daily, if not more often - then you don't know if the customer is getting what they want.

I've worked on various projects that do some of these - short iterations, continuous integration for unit testing, regular releases to the customer, even pair programming; but I have never worked on a true agile project. It's just not possible to do true agile software development if you don't have complete buy-in and ownership from the entire team - from the customer down to the individual developer. I dream of the day that I'll have a chance to work in an environment like this.

The Future of Agile

Agile engineering, Agile process... how does this all tie together? Where is the knowledge moving and how do we get there? Personally, I'm on board with Behavior Driven Development group. Imagine taking every aspect of Agile, and wrapping it up into one very cohesive set of processes and practices. I get all kinds of excited just thinking about it.