I read Sergio’s post on “Javascript, time to grok closures” today and it was very enlightening. Overall, it helped me to understand closures more than I previously had – not just in Javascript, though. I put together a quick sample on closures in C# to illustrate the same behavior that Sergio is talking about in the ‘Closures can be tricky’ section of his post. I’m happy to see C# is behaving the same way as Javascript, in the case of ‘value’ types in closures. This probably isn’t new to anyone that understands closures already. It’s new to me, though, and seems to be a fairly important concept to understand when using anonymous methods (lambda expression or otherwise) and closures.

Illustrating The Closure Scope For Value Types

Here is the sample code I put together, in unit test form, to illustrate the same behavior that Sergio is pointing out.

[TestFixture]

public class When_referencing_a_value_type_variable_from_the_parent_scope_directly_in_a_closure : ContextSpecification

{

   private readonly IList<Func<int>> theActions = new List<Func<int>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           Func<int> foo = (() => { return i; });

           theActions.Add(foo);

       }

   }

   [Test]

   public void Should_reference_the_original_value_in_the_parent_scope()

   {

       //since the value of "i" in the original scope is now 4 (having gone through all of the loop's iterations)

       //every value in the execution of the "foo" anonymous function will be 4

       int expectedValue = 4;

       int value0 = theActions[0]();

       Assert.AreEqual(expectedValue, value0);

       int value1 = theActions[1]();

       Assert.AreEqual(expectedValue, value1);

       int value2 = theActions[2]();

       Assert.AreEqual(expectedValue, value2);

       int value3 = theActions[3]();

       Assert.AreEqual(expectedValue, value3);

   }

}

[TestFixture]

public class When_passing_a_value_type_variable_from_the_parent_scope_to_a_method_called_by_a_closure : ContextSpecification

{

   private readonly IList<Func<int>> theActions = new List<Func<int>>();

   private void dooFoo(int value)

   {

       Func<int> foo = (() => { return value; });

       theActions.Add(foo);

   }

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           dooFoo(i);

       }

   }

   [Test]

   public void Should_create_a_copy_of_the_variable_and_retain_the_original_value()

   {

       //since we called a method, passing "i" from the original context into that method,

       //the copied value was sent to the "foo" anonymous function, meaning that we got all

       //of the numbers - 0, 1, 2, and 3 - in the anonymous "foo" functions.

       int expectedValue = 0;

       int value0 = theActions[0]();

       Assert.AreEqual(expectedValue, value0);

       expectedValue = 1;

       int value1 = theActions[1]();

       Assert.AreEqual(expectedValue, value1);

       expectedValue = 2;

       int value2 = theActions[2]();

       Assert.AreEqual(expectedValue, value2);

       expectedValue = 3;

       int value3 = theActions[3]();

       Assert.AreEqual(expectedValue, value3);

   }

}

Illustrating The Closure Scope For Reference Types

It gets a little more interesting when dealing with reference types. The same basic statements can be made for reference types, but we have to remember that we’re now dealing with what are essentially pointers, not values. So, when a reference type variable gets copied, it still points to the same object on the heap. Thus, when dealing with reference types in closures, the scope of the variable that creates the reference type suddenly becomes much more important.

To illustrate the important of variable scope when dealing with closures, I’ve created some examples that scope a small reference type (class) in a few different ways. I’ll use this class definition as the reference type in all three examples.

public class SomeClass

{

}

Closure Of A Reference In Parent Scope

[TestFixture]

public class When_creating_a_closure_for_a_reference_that_is_part_of_the_parent_scope : ContextSpecification

{

   SomeClass someClass;

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           someClass = new SomeClass();

           theValues.Add(someClass);

           Func<SomeClass> foo = (() => { return someClass; });

           theClosureValues.Add(foo);

       }

   }

   [Test]

   public void Then_all_closures_should_reference_the_last_instance_of_the_variable_from_the_parent_scope()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreNotSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreNotSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreNotSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this example, we’ve declared the “someClass” variable outside of the for-loop. This creates a variable that is scoped outside of the loop, and in this case outside of the Context() method. As long as this variable is scoped outside of the for-loop, we will only have a single variable to create our closure on. We see that the foo() anonymous method encloses someClass, causing it to stay in scope. What we don’t see, though, is that the foo() anonymous method will not be evaluated until the observations (just before the asserts) are executed. This means that every foo() method will have a reference to the same instance of “SomeClass”, not the individual instance that was created inside of the for-loop.

Closure Of A Reference That Is Scoped Within The Loop

[TestFixture]

public class When_creating_a_closure_for_a_reference_that_is_scoped_within_the_forloop: ContextSpecification

{

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           SomeClass someClass = new SomeClass();

           theValues.Add(someClass);

           Func<SomeClass> foo = (() => { return someClass; });

           theClosureValues.Add(foo);

       }

   }

   [Test]

   public void Then_all_closures_should_have_their_own_reference_from_the_loop_iteration()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this example, we’ve declared the “someClass” variable inside of the for-loop, and is thus, in the local scope of the for-loop. Since the foo() anonymous method is creating a closure on a variable that is scoped to the for-loop, we can be assured that each foo() method will point to the “SomeClass” instance that was created in the for-loop. Our closure’s scope is now limited to the for-loop.

Closure Of A Reference Pointer Copy

[TestFixture]

public class When_creating_a_closure_for_a_copy_of_a_reference_pointer : ContextSpecification

{

   SomeClass someClass;

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   private void dooFoo(SomeClass value)

   {

       Func<SomeClass> foo = (() => { return value; });

       theClosureValues.Add(foo);

   }

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           someClass = new SomeClass();

           theValues.Add(someClass);

           dooFoo(someClass);

       }

   }

   [Test]

   public void Then_all_closures_should_have_their_own_reference_to_the_copied_reference_pointer()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this last example, we can see that the “someClass” variable is again scoped outside of the Context() method. However, notice that we are now calling a “dooFoo()” method inside of the loop, instead of creating the closure locally. The effectively means that we are not creating a closure of “someClass” anymore. When someClass is passed into the dooFoo method, a copy of the reference is made – that is, we have a new variable with a new scope; this new variable just happens to be pointing to the same object as someClass, on the heap. Since we are now creating a closure around the “value” parameter of dooFoo, the scope of the closure has changed and acts the same as the closure of a reference in that is scoped with the loop. Each foo() anonymous method, with it’s closure, will now have it’s own instance of “value” to reference – the reference that was created in the scope of the for-loop, and then copied into the “value” parameter.

In “Closing” (pun intended )

It’s important to keep the scope of the variables in our closures in mind. If we scope our closures incorrectly, we can have some strange bugs in our system. Understanding the scope of your closures does require us to know a little bit of computer science, though – stack vs. heap, reference vs. value type – but that’s not a bad thing.

(If you see something wrong in my explanations or think my spec/observation names should be changed, let me know. I tried to make it clear, but am not sure that I did a good job.)

Update: There’s a wealth of knowledge out there that I just haven’t been aware of until now. Thanks to the many commenters of this post, other posts, and other conversations for the links and info. Here is a list of items that I’m finding to be very helpful in understanding what is only a surface level “validation” problem:

• Validation in a DDD World
• A response to Validation in a DDD world
• DDD: Entity Validation and Command/Query Separation
• DDD: CQS and Bounded Contexts
• Greg Young Discusses State Transitions in Domain-Driven Design and DDD Best Practices (InfoQ Video interview)
• DDD: Command Query Separation as an Architectural Concept

Update 3/3/2009 – additional info in the continuing conversation.

• Who Knew Domain Validation Was So Hard?
• Validation and Domain Driven Design

Update 3/11/2009

• DDD: Validity, Consistency and Immutability
• DDD: Invariants or Contextual Validation?

There’s a whole host of stuff out there linked from these articles and posts, as well. It’s obvious now that I have a lot to learn and think about, and I think I have a better sense of where I need to start looking. I also think that the ideas Sean Biefeld expressed about never letting an entity be in an invalid state are more reachable than I previously thought.

----------------------------------

This post started as a comment in Sean’s post on Entity Validation Ideation but quickly grew. I’m hesitant to post separately because I believe the conversation should continue along the lines of what he was originally saying. There’s just so much junk tossing around in my head, though, I didn’t think it would be appropriate to post a 50 page diatribe in a comment.

A while back, I asked the question “Where does required info validation belong for an entity?” There were a lot of good answers to that question and my team and I took some very interesting directions based on those answers. However, I still haven’t had the ‘validation’ question completely answered in my mind. Fortunately, I’ve got the brains of my fellow Los Techies (and coworker in this case), to keep me from giving in to the entropy that wants to keep me mediocre.

I love the theory that Sean is talking about - especially when stated as 'proactive' vs 'reactive'. I've had a hard time with this in practice, though. In the last few weeks – days, really - it seems to me that we need to have both proactive and reactive validation in our applications.

Being Proactive

The domain model that I create wants to prevent an object from being invalid, so I require a minimum set of information in my constructors, etc etc. This helps me prevent an object from ever being created in an invalid state. this is the proactive approach. I love this approach and I use it a lot.

Oh Wait, I Need Reactive

The proactive recently failed me, though. I had ‘the perfect’ domain model with the all the validation and rules composed in my entities, preventing the information from being added to the model if it was not valid. However, when I got to the user interface, I realized that I had a major design problem. The user interface is not 'all or nothing' scenario. The user is allowed (expected, really) to only provide one field of information at a time. After all, you can't type into more than one text box at a time - unless you have multiple keyboards and multi-input capable systems. So, when a user is adding information to the inputs of the form, we need to have a reactive approach. We need to provide real time validation of what the user is putting into the system, so that they can be immediately notified of any errors in what they are providing. This reactive approach is needed because the user interface is entirely reactive to the user's actions.

Proactive AND Reactive?

At the same time that we need both a proactive and a reactive solution, we want to adhere to the "Don't Repeat Yourself" principle, encapsulating the validation logic into the appropriate location(s). If we require reactive validation in the user interface, and proactive validation in the domain model, how do we avoid breaking DRY?

On the surface, proactive and reactive validation seem to be at odds with each other – at least, they do in my mind. I don't think they need to be, or should be, though. We just need to find the higher level order in which we can enable both. Unfortunately, I haven't been able to do this yet – at least not in a way that really seems to satisfy me. I'm curious to know how others approach this situation. How do we balance this triad of need: Proactive, Reactive, and DRY?

I need some fresh perspectives on this, and I think Sean is starting out with some great ideas. Now if I can just learn to throw away my own bias to how I’ve already been handling this, maybe I’ll be able to learn something.

I found the Motivator this morning. It lets you create your own motivational pictures. So, here’s my first run at creating the SOLID software development principles in motivational picture form. I ‘borrowed’ the images from google image search results. I hope you find them to be as fun as I do! I have them all hanging up in my team room, already.

(Update: I never expected the response to this post to be so great! There have been many inquiries about prints, high res versions, etc. As I said in the comments below, the source pictures that are linked via the thumbnails are all I have. There does appear to be some effort to produce high res versions, though. See the comments for more info!)

Update: Due to the continuous request for prints, posters, calendars, etc, we (LosTechies) are looking into what it would take to get these turned into high quality prints of various types. I don’t have any detail yet, but I am hoping to have some good info on this, fairly soon.

SOLID

Software development is not a Jenga game.

(This one was created by Mark Nijhof’s. He posted it via twitter and I’m borrowing it for my own collection.)

Single Responsibility Principle

Just because you can, doesn’t mean you should.

(Update: I knew I had seen this Swiss Army knife in a Single Responsibility post before. Gabriel reminded me where.)

Open Closed Principle

Open chest surgery is not needed when putting on a coat.

Liskov Substitution Principle

If it looks like a duck, quacks like a duck, but needs batteries – you probably have the wrong abstraction

Interface Segregation Principle

You want me to plug this in, where?

Dependency Inversion Principle

Would you solder a lamp directly to the electrical wiring in a wall?

I’ve written a lot of specification tests like this in the last three years, from a UI / Workflow perspective, with Model-View-Presenter as my core UI architecture:

[TestFixture]

public class When_starting_some_process()

{

 IMyView view;

 MyPresenter presenter;

 [Setup]

 public void Setup()

 {

   //...setup code and execute stuff for the test here

   view = MockRepository.GenerateMock<IMyView>();

   presenter = new MyPresenter(view);

   presenter.StartSomeProcess();

 } 

 [Test]

 public void Should_attach_the_view_to_the_presenter()

 {

   presenter.View.ShouldNotBeNull();

 }

 [Test]

 public void Should_show_something()

 {

   view.AssertWasCalled(v => v.ShowSomething(something));

 }

}

The Devil In These Details

I had one of those ‘aha!’ moments yesterday where several of my nagging suspicions and annoyances at writing specification tests like this example finally gelled into a coherent understanding. That understanding is easily stated by saying that the test, “Should_attach_the_view_to_the_presenter” is invalid and should never be written. There are a number of reasons for this.

1. In practicing BDD, the technical jargon that is leaking into the test is irrelevant to the real value that is intended with this specification
2. In practicing any form of TDD, the implementation detail of attaching a view to a presenter creates a brittle test – I care about the API and how the system really works, not a very technical, implementation detail like this.
3. Exposing the “View” property of the Presenter object is a violation of encapsulation and an ‘over-intimate’ smell. My test has far too much fine detail, granular knowledge of what’s going on behind the scenes, to really be of any practical value
4. Finally – and possibly outweighing all other reasons combined – it’s simply not necessary.

Look at the second test: “Should_show_something”. It’s actually using the view. Presumably, the “StartSomeProcess” method on the presenter is going to call a method on the view – that’s why we are asserting that the method on the view was called. If this is true, then it can be safely assumed that not having a view attached to the presenter would throw a null reference exception when trying to call that method on the view.

If not having the view attached to the presenter results in a null reference exception for the other test, then we have a valid reason for that test to fail. We certainly can’t say that the view’s method was called when the view is null. Therefore, testing to ensure that we have a view attached to the presenter is a duplication of effort. We’re only proving what we have already proved transitively, via another test.

Beauty And Meaning In Simplicity

Assuming that this is all true, we can remove that test entirely. Our specification now looks like this:

[TestFixture]

public class When_starting_some_process()

{

 IMyView view;

 MyPresenter presenter;

 [Setup]

 public void Setup()

 {

   //...setup code and execute stuff for the test here

   view = MockRepository.GenerateMock<IMyView>();

   presenter = new MyPresenter(view);

   presenter.StartSomeProcess();

 }

 [Test]

 public void Should_show_something()

 {

   view.AssertWasCalled(v => v.ShowSomething(something));

 }

}

It’s one less test to deal with, yet is as expressive and meaningful. In fact, I would say it is more meaningful because we have avoided all of the problems I listed. I’m reminded of a quote by Alan Cooper:

“No matter how beautiful, no matter how cool your interface, it would be better if there were less of it.”

Although I think he was specifically addressing User interfaces in this quote, it is applicable to all interfaces – programmatic, user, etc. Simplicity and subtleness is the key. I know I’m not the first person to talk about this problem, why it’s a problem, or the solution. I’m only claiming that I finally had that ‘aha!’ moment and realized why so many others have talked about this before.

Let's assume that we are doing the appropriate amount of testing during our development process. If we include TDD, test automation, test engineers and customer acceptance testing, we should find the majority of the bugs in our system before they are released. However, not every bug will be found. There will be some situation that no one thought about before. There will be some special circumstance on someone's computer that hasn't been accounted for. There will be some client data that does fit the expected variance, despite the data being valid. The point is, there will be something that breaks after we deliver the software. Worse yet - it doesn't even take delivery to find bugs. What happens when the software gets to customer acceptance and the customer says that something is wrong, broken or whatever? We simply must account for the inevitable bug fixes and emergency patches in our system.

Our current kanban board, with all it's pipelines and queues, hasn't addressed the need to address problems. But before we get into any changes to the board, we have to define and create the culture of quality that we need in our team.

A Culture Of Quality

There are many different aspects of quality and many different ways to view and interpret quality. The individual specifics, as always, come down to your team, the project and the needs of both. Even with these variances, though, there are a few key concepts that should be found in any culture of quality, including Whole Team / Collective Ownership and Zero Defects.

Collective Ownership

I've talked about this concept before (see above link) and would recommend reading those prior posts. One thing I would like to add, though is that collective ownership can only success when the team takes their individual egos out of the equation. We must be able to accept criticism as a way to improve ourselves and our team. When we let go of our ego, we can be honest with ourselves and others, enabling everyone to own every part of the system.

Zero Defects

This is a subject that I have not talked about before, but is prevalent throughout the continuous improvement philosophies that I subscribe to. Wikipedia lists four principles of Zero Defect methodologies, all of which are paramount to our culture of quality.

1. Quality is conformance to requirements
2. Defect prevention is preferable to quality inspection and correction
3. Zero Defects is the quality standard
4. Quality is measured in monetary terms – the Price of Nonconformance

Don't be confused by "requirements" in item number one, though. In the world of software development, the word "requirements" has many meanings - not simply the feature list or functional descriptions set forth by our customers. Our teams and processes have certain requirements that are imposed on top of, and into each of the business requirements for the software. These often include the use of source control systems, test driven development, principles like S.O.L.I.D., etc.

Quality Checks in Kanban

Collective ownership and zero defects are only the tip of the iceberg. There is so much more to a culture of quality and so many different aspects of quality to account for. However we define quality, though, our ability to create a culture of quality is necessary for our kanban system to work. Without it, we lose the continuous improvement and elimination of waste that defines Lean. With it, though, we can introduce some specific tools to our process and improve our kanban system by having it handle the errors that are found in our systems.

Andon

From Wikipedia:

"a system to notify management, maintenance, and other workers of a quality or process problem"

When an issue is found in our system, we need to notify the team immediately so that the problem can be taken care of. The idea in a manufacturing line is to allow a worker to stop the line when a problem is found. In our grocery store example, andon could be invoked by a shelf stocker noticing a loose nut or bolt on the shelf and then notifying a maintenance person. In software development, andon can take many different forms. We report our issues list during our daily standup. We create issue tickets in our issue management system. If we're lucky and work in a company that supports the idea of a team room, we may just need to look up or turn around and let the team know that we have problems. We could even place an actual red flag on the desk of our developers, testers, customer representatives, etc., and have them raise that flag when they see a problem. Any of these ideas, plus many many more, can all be our andon system.

Andon itself is not intended to be an all encompassing electronic system with metrics and reports and yadda yadda yadda. It needs to be simple. It needs to be easily employed by anyone on the team. And it needs to mean something to the team. If a team member throws their andon card out on the table, the culture of that team needs to be ingrained with the knowledge that work may stop until the problem is addressed.

There's so little to andon, yet so much more than what I've described. However andon is enabled, it is critical to our zero defect policy in software development.

Jidoka

AKA "Autonomation", AKA "Automation with a human touch", AKA "intelligent automation", AKA ...

From Wikepedia:

"Autonomation prevents the production of defective products, eliminates overproduction and focuses attention on understanding the problem and ensuring that it never recurs."

I've talked about Jidoka in previous posts and would recommend reading them, at this point.

• Jidoka - If it's broke, fix it now
• Jidoka - More than just 'if it's broke, fix it now" - fix it and inform the team

I don't have much else to add, other than to say that Jidoka and Andon go hand in hand. Automated build servers (such as CCNet, TeamCity and many others) can often combine Andon and Jidoka for us by giving us instant visual feedback when something is broken. I've also recently set up BigVisibleCruise in my team area, giving even the casual observer the knowledge of whether our builds are broken or not.

Applying Andon and Jidoka to Our Kanban Board

Once we have the concepts of Andon and Jidoka in our team and culture, we can use these tools to generate issue cards and then look at a few possible changes to our kanban board to account for them. The three basic methods that I have seen used include:

1. Creating an Emergency Fixes pipeline
2. Tacking a smaller bug notice onto an existing card
3. Putting a Bug card in the backlog

I'm sure there are other alternatives, too. In my current team, we use the first and third method and are considering the second one as well. 

Options two and three essentially require no change to our kanban board. Implementing option two is a distinctive way of visually attaching a bug notice to one of the cards in our system. This could be done with little bug stickers, little red cards, or any other visual indicator that the team agrees on. Option three also needs something distinctive. Since we are creating an entire card for the bug, though, the entire card should be distinctive. I would recommend using a card that is colored red to signify issues. I would also recommend prioritizing the bug to the top of the backlog queue, when using option three. This will ensure that the bug gets worked as quickly as possible.

Option number one can also make use of number two and/or three. When we move a card into the Emergency Fixes pipeline, we may want it to be distinguishable as an issue by tacking on our bug symbol or by using a colored card.

Emergency Fixes Pipeline

Depending on the complexity or severity of the bug, we may want to include Analysis and Customer Acceptance in our Emergency Fixes. With andon and jidoka in mind, we will want to ensure that we fix any emergency issue immediately. This is not always possible, however. The customer may decide to delay the fixing of an issue for whatever reason. This leads us to only needing a single pipeline for Emergency Fixes, letting us set our limit to one.

We can easily add an Emergency Fixes pipeline to our kanban system, placing it directly underneath our existing WIP pipeline. This special pipeline can be designated with a name, color code, or other marks as needed.

If you are supporting production releases, you may also need to include a Delivery queue specifically for emergency fixes. This would allow multiple fixes to be compiled into a single patch release. There are other configurations to this, of course. As always, you will need to find what your specific team needs and create your process to suit.

Where Do We Go From Here?

With an Emergency Fixes pipeline in place, our kanban system is now set up to handle just about every situation that we will encounter. However, this does not mean that our system is perfect or truly complete. No process, no matter how well defined it is, is worth anything if the people running the process do not believe in it. I've said it before and I'll continue to say it - never stop improving your process. Always be mindful of waste, friction, smells, problems or whatever you want to call it. Inspect, adapt and continuously improve your process. Perfection is a journey, not an end-goal.

Stay tuned to my Adventures in Lean series, as I continue to explore the various aspects of lean software development in my own team and company culture.

In the comments of my previous post - Descriptive State Enumeration - Maxim Tihobrazov asked me to show how to map a state pattern with NHibernate; and I am more than happy to oblige!

NHibernate Mapping Options

I certainly don't claim to be an expert on NHibernate, but I do use it on a daily basis and I've solved my fair share of problems - including how to map a State pattern with NHibernate. According the NHibernate inheritance documentation, there are three core ways of mapping this pattern:

1. Table per class hierarchy (I've always called it "record per class") - each subclass (state instance, in this case) is associated with a specific record in a table. All of the subclasses are found in the same table.
2. Table per subclass (also called a "joined sub class") - each sub class has it's own table and is joined to the super class by a one to one relationship with it's primary key.
3. Table per concrete class - each sub class has it's own table and is mapped by the specific class type, not by the abstraction.

My Choice - Table Per Class Hierarchy

In my current set of applications, I've always used the table per class hierarchy option so that is what I will describe here. My reasons for choosing this option are simplicity in the mapping and more importantly - not having a need to store the records in their own tables. Each of my states can be very cleanly represented as a record in my state table.

Setting Up Shop

Let's consider the same coffee shop that I've used in both of my PTOM posts - in this case, we're dealing specifically with the Order class and it's OrderStatus state. It's necessary to note that when I am working with NHibernate and the state pattern, I actually do set up my state models a little different. First off - we need an Id for NHibernate to map to the table's primary key. Secondly, I like to keep the core information in the abstract base class and provide all the varying values via a constructor. Third, each of the inheriting classes needs to provide a default (no args) constructor. Forth, we need a way for NHibernate to know which class it should actually instantiate, when loading - a "descriminator". I like to use a simple string Name property for this. And lastly - I think (though I am likely wrong on this aspect) that the abstract base class needs to provide a default constructor. Fortunately, the constructors we need don't need to be public - they can be private.

With all of that being said, here is realistic code base that I would use for my Order and OrderStatus model:

public class Order

{

   public int Id { get; set; }

   public OrderStatus Status { get; set; }

}

public abstract class OrderStatus

{

   public static InProcess = new InProcessStatus();

   public static Totaled = new TotaledStatus();

   public static Tendered = new TenderedStatus();

   public static Delivered = new DeliveredStatus();

   public int Id { get; set; }

   public bool DisplayForFullfillment { get; }

   public string Name { get; set; }

   private OrderStatus() { }

   private OrderStatus(int id, bool display, string name)

   {

       Id = id;

       DisplayForFullfillment = display;

       Name = name;

   }

}

public class InProcessStatus: OrderStatus

{

   private InProcessStatus(): base(1, false, "InProcess");

}

public class TotaledStatus: OrderStatus

{

   private TotaledStatus(): base(2, true, "Totaled");

}

public class TenderedStatus: OrderStatus

{

   private TenderedStatus(): base(3, true, "Tendered");

}

public class DeliveredStatus: OrderStatus

{

   private DeliveredStatus(): base(4, false, "Delivered");

}

The Fluent NHibernate Maps

I'm a huge fan of Fluent NHibernate. I've been using it since just after it was branched from the original ShadeTree code. And at one point, I had submitted so many patches that I was made a comitter on the project. So, it should be no surprise to anyone that I'm going to advocate using FluentNHibernate over the .hbm.xml mapping files. Truth be told, I don't even remember how to do the needed descriminators in hbm.xml files. There was some trick to the descriminator being the first specified items after the Id or something... it's just easier to do in FluentNHibernate, so I don't bother with hbm.xml files anymore.

The Order map is going to be as standard as any other map. You don't need to do anything special here. Just map the Id of the Order and the referenced OrderStatus. (I like to add a "CreateMap()" method that is called from the constructor, so that I can avoid the "virtual method call from a constructor" warning. But I also use the "treat warnings as errors" option for my C# projects).

public class OrderMap: ClassMap<Order>

{

   public OrderMap()

   {

       CreateMap();

   }

   public void CreateMap()

   {

       DefaultAccess.AsProperty();

       WithTable("Orders");

       Id(o => o.Id).GeneratedBy.Assigned();

   }

}

It's really the OrderStatus map that is special in this case. This is where we get into the details of the descriminator - telling NHibernate which specific instance to create, when loading the data from the database. In our case, we have added a "Name" field to our OrderStatus object and we will be explicitly using this property as the descriminator.

public class OrderStatusMap: ClassMap<OrderStatus>

{

   public OrderStatusMap()

   {

       CreateMap();

   }

   public void CreateMap()

   {

       DefaultAccess.AsProperty();

       WithTable("OrderStates");

       Id(s => s.Id).GeneratedBy.Assigned();

       DiscriminateSubClassesOnColumn<string>("Name")

           .SubClass<InProcessStatus>()

               .IsIdentifiedBy(OrderStatus.InProcess.Name)

               .MapSubClassColumns(x => { })

           .SubClass<TotaledStatus>()

               .IsIdentifiedBy(OrderStatus.Totaled.Name)

               .MapSubClassColumns(x => { })

           .SubClass<TenderedStatus>()

               .IsIdentifiedBy(OrderStatus.Tendered.Name)

               .MapSubClassColumns(x => { })

           .SubClass<DeliveredStatus>()

               .IsIdentifiedBy(OrderStatus.Delivered.Name)

               .MapSubClassColumns(x => { })

       Map(s => s.Name);    

   }

}

The key to all of this is the DescriminateSubClassesOnColumn method. The generics <string> tells us what .NET Type is being used to identify the specific class instances that we are dealing with and the ("Name") parameter is the column name in the database that represents the specific instance.

Then, we have the specific class instances referenced by the ".Subclass<type>" calls. This is a fluent interface, so ".Subclass" is a method that gets called directly off the DescriminatoeSubClassesOnColumn method. The specific type we want NHibernate to know about is specified in the generics parameter: ".SubClass<InProcessStatus>". The "IsIdentifiedBy" is where we give NHibernate the knowledge of what value in the "Name" column maps to what specific class in our code. To prevent magic string syndrome from setting in, I like to use the actual enumeration pattern of my OrderStatus to specify the string name of the class.

And finally, we have to provide an ugly workaround for our maps via the "MapSubclassColumns" method. There is an implementation issue in FluentNHibernate currently, and because of this issue we are forced to call the "MapSubclassColumns" method, with an empty lambda expression. If we don't call this method, the sub class will not get registered and NHibernate will not know how to handle the data in question. (I am hoping to fix this issue at some time, and make it so we don't have to call that method. I just haven't had time recently.)

Wrapping Up The Map

The rest of the map (the one remaining "name" property being mapped, in this case) is a regular NHibernate map. You'll notice, though, that we never mapped the "DispalyForFullfillment" property. I explicitly choose to leave any and all "volatile" properties out of my state maps. By doing this, I am able to add, edit, and remove any properties or methods on the state objects that I need, without having to change the NHibernate mappings. Since the state objects I am dealing with don't change without recompiling the code anyway, I don't need the ability to define them in the database. However, if you do need or want the flexibility of defining your states in the database, you can map each individual property of the state. Just remember that you will have to modify both the code and the database, to make the changes complete.

I hope this quick look at mapping a state pattern with Fluent NHibernate will help to shed some light on the subject, for someone. I realize that I have only provided the FluentNHibernate mapping, though. I specifically chose to do this because it would be a serious chore for myself to create a code project with NHibernate set up so that I can actually verify my mapping xml is correct. However, the translation from FluentNHibernate back to standard .hbm.xml files should be fairly straightforward, with the help of the NHibernate Documentation. If anyone out there is willing to help out and send me the correct .hbm.xml mapping, I would be more than happy to add it to this post and credit you with the work.

This post is part of the November 2008 Pablo's Topic Of The Month (PTOM) - Design Patterns and will primarily outline the State pattern, with an Enumeration or Descriptor pattern thrown in for good measure.

Switch statements and if-then statements are not object oriented code. They are conditional logic for procedural control and processing. This doesn't mean that they are bad - by no means. It only means that they are not object oriented. There are many legitimate reasons for using if-then statements and a few legitimate reasons for using switch statements. However, there are also many bad reasons for using these statements.

Setting Up Shop

Before we get into the guts of the State pattern, let's consider the Point of Sale system of our coffee shop again. When a customer is placing an order, we need to track what they are ordering. This would likely be done through an Order object. While the customer is placing their order, the order is considered "in process". Once the customer is done with their order, the cashier can "total" the order. Then, the order is "tendered" (paid for). And finally, the order is "delivered" to the customer. Each of these quoted items is a state that the order goes through during it's brief run through the coffee shop. In C#, it would be very easy to represent these states with a simple enum:

public enum OrderStatus

{

   InProcess = 0,

   Totaled = 1,

   Tendered = 2,

   Delivered = 3

}

With this enum in hand, we can have our point of sale system set the correct order status according to what is going on at the moment.

//when a coffee is ordered

Product regularCoffee = new Product("RegularCoffee", 2.99);

Order order = new Order();

order.Add(regularCoffee);

order.Status = OrderStatus.InProcess;

//when an order is totaled

double total = order.GetTotal();

order.Status = OrderStatus.Totaled;

//when the customer pays for the order

order.AmmountPaid = ammountPaidByCustomer;

order.Status = OrderStatus.Tendered;

//and finally, the order is delivered

order.Status = OrderStatus.Delivered;

Now this code we have so far is technically correct - it works and it gets the job done. Unfortunately, though, this code will likely lead us down a bunch of dark paths of code duplication, ugly switch statements, and an unmaintainable mess. For example, what happens when we start integration our Order object into the rest of the coffee shop and we need to display orders to fulfill on a screen for the order runners to fill. In such a case, we only want to show orders that have been tendered but not delivered, and to try and get orders out the door faster, we'll also show orders that are totaled.  If we were going to examine a single order to determine whether or not we wanted to show it, we would likely have one of the two following pieces of code in place (yes, I know this could be done with a database query. The point is that somewhere in the code you will likely end up with switch statements or if-then statements like this):

//one way to know

bool shouldShowOrder = false;

if (order.Status == OrderStatus.Totaled || order.Status == OrderStatus.Tendered)

     shouldShowOrder = true;

//another way to know

bool shouldShowOrder;

switch (order.Status)

{

   case OrderStatus.Totaled:

   {

       shouldShowOrder = true;

       break;

   }

   case OrderStatus.Tendered:

   {

       shouldShowOrder = true;

       break;

   }

   default:

   {

       shouldShowOrder = false;

   }

}

//after we know, we show it

if (shouldShowOrder)

     ShowTheOrderForFulfillment(order);

So - what happens when the order status list change? Or if we now have a need to show a different order status on our display? We would have to find all of the locations that used this status enum and check to see if it needs to be changed, to handle the new status or change in how the status is handled. This is where the problems really start - hoping that you got all of the uses changed and all of the right states handled in the right ways. Fortunately, there's a design pattern that we can use to simplify this situation and eliminate many of the associated switch and complex if-then statements.

The State Pattern

Wikipedia describes the State Pattern as "a clean way for an object to partially change its type at runtime." The C2 Wiki gives a little more insight into the State Pattern, as well: "Allow an object to alter its behavior when its internal state changes. The object will appear to change its class."

What both of these descriptions are really getting at, is that a single concept in the real world is likely going to be modeled as more than one class in code - composed of many different pieces. When any one part of a concepts model changes, the state of that concept is changed. The relationship between our Order class and OrderStatus enum are a prime example of this composition. What the state pattern allows us to do is change the behavior of our modeled concept by partially changing the composition of our model.

Modeling Our State As State Model

To model the state of our Order with a state pattern instead of a state enum, and begin introducing the ability to change our systems behavior without changing the code that relies on the state, we need to introduce an abstraction that can represent any state we care about. To start with, we need to know what orders should be shown on our display for the order runners.

public interface IOrderStatus

{

   bool DisplayForFulfillment { get; }

}

After introducing this abstraction, we can introduce any state implementation that we need.

public class InProcessStatus: IOrderStatus

{

   public bool DisplayForFullfillment { get { return false; } }

}

public class TotaledStatus: IOrderStatus

{

   public bool DisplayForFullfillment { get { return true; } }

}

public class TenderedStatus: IOrderStatus

{

   public bool DisplayForFullfillment { get { return true; } }

}

public class DeliveredStatus: IOrderStatus

{

   public bool DisplayForFullfillment { get { return false; } }

}

Once we have these states in place, we can replace our code that sets the state with these objects.

double total = order.GetTotal();

order.Status = new TotaledStatus();

And we can also significantly reduce the code that needs to know if we should display this order in our order display system. Now, instead of having to do those if-then or switch statements, we can simply check the order status:

//the order class now relies on the IOrderStatus for it's composition

public class Order

{

   public IOrderStatus Status { get; set; }

}

Yes, we are still using an if-then statement. However, this one simple statement is no longer an issue in terms of maintainability. We don't have to change this code as we add, remove, or change the states of our system. Since we only depend on the IOrderStatus interface for our composition now, we can freely change the implementation of the states as we need - change the composition of the Order's model.

But Wait! There's More!

We don't have to stop at simple boolean values on our state abstraction - we can provide actual behavior via methods in our state. And the best part is - when we can add and remove any behavior or boolean values, or whatever else we need to change with the state of our Order, we don't have to change any of the existing code that uses the existing DisplayForFullfillment value. This property is defined on the interface, letting us use it where we need it and ignore it where we don't need it.

By using a state pattern, as we've seen here, we have introduced the ability to change the behavior of our system without having to change the places that need to know about the state of the order. We've gained a great level of encapsulation. We've gained flexibility in changing states when we need to - and we have an abstraction that hides the details when we don't need them. Seems like a winning situation to me.

But Wait! There's Still More!

Unfortunately, one thing we lose is the friendly syntax of our enumeration. For example, "OrderStatus.Totaled." Personally - I like the simplicity that enumerations offer us in our code. Something about saying "OrderStatus.Totaled" seems to be right to me. So how do we marry the state pattern to the enumeration syntax? The answer is simple - an Enumeration pattern (also called a Descriptor pattern).

We can make a simple change to our order status abstraction - make it an abstract base class. Then we can provide some static fields to provide access to the class instances that we want. To enforce the enumeration like syntax, we can also make the constructor private.

public abstract class OrderStatus

{

   public static InProcess = new InProcessStatus();

   public static Totaled = new TotaledStatus();

   public static Tendered = new TenderedStatus();

   public static Delivered = new DeliveredStatus();

   public abstract DisplayForFullfillment { get; }

   private OrderStatus() { }

}

public class InProcessStatus: OrderStatus

{

   public bool DisplayForFullfillment { get { return false; } }

}

public class TotaledStatus: OrderStatus

{

   public bool DisplayForFullfillment { get { return true; } }

}

public class TenderedStatus: OrderStatus

{

   public bool DisplayForFullfillment { get { return true; } }

}

public class DeliveredStatus: OrderStatus

{

   public bool DisplayForFullfillment { get { return false; } }

}

//the order class now relies on the OrderStatus base class for it's composition

public class Order

{

   public OrderStatus Status { get; set; }

}

And now we have our nice friendly enumeration syntax back!

//when an order is totaled

double total = order.GetTotal();

order.Status = OrderStatus.Totaled;

This post is part of the November 2008 Pablo's Topic Of The Month (PTOM) - Design Patterns and will outline a simple Command pattern, its implementation and use.

One of the core principles of object oriented software development is the idea of Coupling, "the degree to which each program module relies on each one of the other modules" (Wikipedia). When we build software, we should strive to attain low coupling - keep the individual modules of the system as separated as possible. Of course, if there is zero coupling in a system, then the system won't really do much for us. After all, you can't write software that is very useful if you are not allowed to have any dependencies.

Setting Up Shop

Before we break down into the pattern, let's consider the context of a Point Of Sale system at a coffee shop. In this system, we have a menu where your server can punch in your order with all of the options you want and produce an order that is fulfilled somewhere else. When the server starts pressing the menu items and the system begins to compile the order, there is a connection between the menu system that they are using, the various products that have been configured in the system, and the back-end ordering system. This is a rather obvious location where coupling can quickly become high - the UI, the products, and the back-end ordering system could very quickly become a spaghetti mess of tangled dependencies and tight coupling - but we don't want that, do we?

To combat the coupling problems of this situation while still allowing dependencies, we need to introduce various forms of abstraction - simple dependencies that are not specific to any implementation, allowing us to change the implementation when we need to. In the case of a menu - be it a WinForms application, a touch screen point of sale system, or whatever else - a Command pattern is often employed to enable the system's functionality without being directly coupled to the actual menu.

The Command Pattern

Originally outlined by the infamous "Gang of Four", the Command Pattern is described as an object that represents an action - a command that will be executed. Within the context of a command, we have several parts that need to be accounted for.

• First and foremost, there is the actual command object - the action that is executed.
• Second, we have the command invoker - the object that depends on the commands existence, and knows how to execute the command.
• And lastly, we have the target of the invocation - the part of the system that needs to take action when the command is executed.

A Simple Implementation

To facilitate the decoupling of specific modules in our system, our command pattern implementations will be created with a naming convention that represents the objects as commands. For example, a very basic command implementation in C# can be represented as an interface such as this:

public interface ICommand

{

  void Execute();

}

By abstracting our command into an interface, we can provide any implementation we need at runtime. This lets us select a coffee from our point of sale system and have another part of the system actually create and handle the order. A pseudo-complete implementation of a command pattern to order a cup of coffee may look something like this:

public interface ICommand

{

   void Execute();

}

public class MenuItem

{

   private ICommand _menuCommand;

   public Text { get; set; }

   public MenuItem(string menuText, ICommand menuCommand)

   {

       Text = menuText;

       _menuCommand = menuCommand;

   }

   public void Click()

   {

       _menuCommand.Execute();

   }

}

public class RegularCoffeeCommand: ICommand

{

   private SomeOrderingSystem _orderingSystem;

   public RegularCoffeeCommand(SomeOrderingSystem orderingSystem)

   {

       _orderingSystem = orderingSystem;

   }

   public void Execute()

   {

       Product regularCoffee = new Product("RegularCoffee", 2.99);

       _orderingSystem.PlaceOrder(regularCoffee);

   }

}

//... somewhere in the application UI

MenuItem regularCoffeeMenuItem = new MenuItem("Regular Coffee", new RegularCoffeeCommand());

//... and when the menu item is clicked

regularCoffeeMenuItem.Click();

The real power of this implementation is that we have completely decoupled our menu system from any specific knowledge of the product ordering system in our coffee shop. All our menu item needs to know about is the ICommand interface. At the same time, our back-end implementation also knows about the ICommand interface - but the back-end also knows about the actual product ordering system. This allows us to create an implementation of the ICommand interface that knows how to invoke our target system. The end result is that we can independently vary the menu system for ordering coffee and our back-end product ordering system.

More Complexity And More Flexibility

Great News! Our command pattern implementations don't have to be limited to such a rigid interface. In fact, my current project has no less than 4 different ICommand interface variations. By introducing the use of generics in C#, we can create some very flexible commands. As an example, consider the following additional interface definitions:

public interface ICommand<T>

{

   T Execute();

}

public interface IParameterizedCommand<V>

{

   void Execute(V value);

}

public interface IParameterizedCommand<T, V>

{

   T Execute(V value);

}

These new interface definitions, in combination with the original definition above, can create a very flexible and very useful set of commands in a system. By creating a "parameterized command" interface, we can provide detail in the command execution that is not available at the time the command is instantiated. And by adding another non-parameterized command with a return value, we can create a system that is able to interact in more complex ways.

With these new command interfaces in mind, let's take another look at our RegularCoffeeCommand. Instead of hard coding the price of the coffee into the command, let's push that knowledge off to another part of the system.

public class RegularCoffeeCommand: IParameterizedCommand<Price>

{

   private SomeOrderingSystem _orderingSystem;

   public RegularCoffeeCommand(SomeOrderingSystem orderingSystem)

   {

       _orderingSystem = orderingSystem;

   }

   public void Execute(Price coffeePrice)

   {

       Product regularCoffee = new Product("RegularCoffee", coffeePrice);

       _orderingSystem.PlaceOrder(regularCoffee);

   }

}

By providing an interface that accepts a parameter, we can move the knowledge of the coffee's price out to another part of the system and provide it at runtime - a database call, a web services call, or anything else we want.

But Wait! There's More!

As you can see, the command pattern can be very useful in helping us decouple our coffee shop's point of sale system from the back-end product ordering system. Some very simple interface definitions can provide a lot of flexibility in how we compose our systems, as well.

What's more - we aren't actually limited to interfaces or base classes for abstracting our commands in .NET. The built in delegate system in .NET provides a great way to encapsulate commands with method pointers. I've previously talked about The Point of Delegates in .NET where I mentioned that delegates provide us with a way of delaying the execution of code. Though it's not strictly an "object" as the command pattern describes, a delegate certainly sounds like a command pattern implementation just waiting to break out of it's skin. In fact, I have created a few command pattern implementations using nothing more than the built in delegates in .NET, several times.

Whether you decide to use interfaces, delegates, abstract classes, or any other implementation method that you can come up with, I hope that you will think about including a command pattern implementation in your systems. It is an easy way of conquering the problems of high coupling between user interfaces and back-ends of the system.

In my last post, I talked about the idea of encapsulation and using it to ensure that our business rules were enforced correctly. What I didn't talk about, though, was the second half of the conversation that my coworker and I had, concerning the patent -> consultation relationship. It turns out that we had the relationship wrong. That's not to say that patients don't have consultations, but that the logical model we were traveling with had an incorrect perspective and was causing us to create some very ugly workarounds in various parts of the system. What really stuck out in my mind, though, was not the idea that we had the model wrong, but how we came to the conclusion of the model being wrong. It has become apparent to me, upon reflection of the conversations and situation as a whole, that design smells are not always evidenced by design related activities, if ever.

A Persistent Problem - Duplicate Redundancy

After some initial coding of the patient -> consultation relationship, we start working on the persistence model via NHibernate. What we have is a patient with a collection of consultations - this is easy to map with NHibernate's one-to-many capabilities. We also have a CurrentConsultation property which needs to be mapped. This property is mapped to the same Consultation table, but only pull one specific consultation based on the business rules that state the current consultation is chronologically the most recent and has not ending date set.

After some thought, we found that there were a few possibilities for handling the CurrentConsultation property in our current model:

1. Create a "CurrentConsultation" object that is mapped to the Consultation table and use a "where" class attribute in the NHibernate mapping that would limit the returned result
2. Create a "CurrentConsultation" object that is mapped to a CurrentConsultation view and have the view coded to return the correct consultation object
3. Add a CurrentConsultationId field to the Patient table, as a foreign key to the Consultations table, and map to the existing Consultation object

After some additional thought, though, we found that each of these solutions has a few significant problems that were going to cause a lot of trouble.

Options 1 and 2

Both of these options have the problem of duplicating business rules into more than one language and location. We would either have the business rules of what constitutes the current consultation in the NHibernate mapping (the 'where' attribute) or in a database view, in addition to the already existing code. Changing the rule would mean changing a minimum of two locations where that rule is handled. This is a bad idea no matter how you look at it.

Both of the options have also created a duplication of knowledge from the concept of a Consultation by creating a "CurrentConsultation" class and a separate NHibernate map for it. We would have the original Consultation class and the new CurrentConsultation class both representing the same data, making an artificial distinction in our code. Again, this is a bad idea. We don't want duplication of these logical concepts. We're also not dealing with a bounded context or any other logical separation of concerns at this point, so there is no need to separate the concept of a consultation into multiple classes.

Option 3

This doesn't appear to have the duplication issue in code, but there is a potential for duplication of data. When we get down to the implementation of NHibernate, we could easily cause duplicate data in the consultation table by saving the current consultation class. We might be able to get around this by not cascading the saves of the current consultation property, but then we'd be forced to ensure the consultation collection was persisted prior to the patient so that we could update the current consultation object's id before saving the patient. Both of these problems sound like a serious pain to me. I'm betting it's possible, but I'm also betting that it would be a nightmare of trial and error to get it right and a lot more code than we should really have to write.

Changing Perspectives

As Joe Ocampo pointed out in the comments of my original post, we had a problem in our system that was really caused by our lack of correct perspective in the situation. Rather than forcing the idea of a patient being the root aggregate in this situation, causing us a lot of headache and frustration in trying to model our persistence layer, a simple change in how we looked at the situation helped us solve the persistence problem and greatly simplified how the application worked.

Joe's comment (with some formatting added):

"One thing I like to challenge developers with when I teach DDD is to flip the aggregate to determine if the model is sound.

I know this is only an example but work with me here.  You indicated you are dealing with a medical system.  We can assume there are certain entities such as Patient, Consultations, Doctor and Practice. In your example you created a model where the patient is the aggregate root for consultations but what if the Doctor simply asked what consultations do I have today?  In this paradigm the Practice is the aggregate root and Consultations are aggregate within where Patient is an aspect of the consultation.  The code would look something like this.

consultations = practiceService(IConsultationService).GetConsultationsFor(doctor);

This also allows the consultation service to encapsulate its own logic for creating a consultation for creating a consultation. You can’t get any closer than that

consultationService.CreateConsultationFor(patient).with(doctor).at(date);

The point I am trying to make is be careful of aggregate roots.  Once you go down that path it is really difficult to back the train up and break it apart."

Though our actual implementation was different, this was the same basic conclusion that we had come to - our perspective on the situation was simply wrong. When we stepped back from the problem and realized that the consultation was the primary focus of the situation, and that a nurse or doctor would be the primary user of that portion of the system, it became rather obvious that our aggregate was in dire need of rework.

A Reflective Perspective

What we ended up with was a Patient object that dealt with all of it's demographics information, billing information, etc, without a CurrentConsultation property or even a Consultations list. Then, on the the separated Consultation object, we added a child property of Patient. Once we realized that our Consultation object was the primary focus and made this distinction in our code, we also realized that the Patient object was carrying far too much information around the system. We found that we actually had two very distinct concepts of a patient, determined by two very distinct bounded contexts.

• In the 'Billing' context, we needed all of the address , billing, and other demographics information about the patient - who they are, where they live, what their insurance is, etc. The existing Patient class filled this need.
• In the 'Consultations' context, we did not need anything from the Billing context, except for the person's name and patient id. What we really care about in the consultations is medical information about the patient - their current prescriptions, allergies, past medical care, etc. So, we created a ' patient' class to represent these needs.

These changed allowed for a much more clearly defined model that was truly reflective of the systems needs. We could easily see the difference between a 'billing' patient and a 'medical' patient, and we were able to code each of these areas of the system without the concerns bleeding into each other. Essentially, we decoupled the system at a module level, not just at a class level.

We also found that the NHibernate mapping problems suddenly went away. Since the Consultation class had a child of Patient, it was a simple many-to-one mapping with no strange sequencing or duplicate data issues. In the screens that deal with the consultation directly, we load the consultation as the aggregate root and go from there. In the screens that need to show patient consultation history, we did a simple query and returned all of the consultations for the given patient. Again, we found a way to decouple our system - this time, at the persistence model.

Design Smells: Not Just A Design Problem

In the end, we were able to recognize a serious design smell in our system - not by the design itself, though. After all, the original code had encapsulated the needs quite well. But, as it turns out, it was a bad encapsulation at a higher level. It wasn't until we started working with the model we had created, specifically trying to persist the model, that we realized our design was not right.

This change was a huge breakthrough for us, not necessarily in the code or the system that was being built, but in how we look at our systems and our domain models. The realization that design smells are often evidenced not by the design itself, but by how the design is used in the infrastructure and other supporting roles of the system, has had a profound impact on how we look at system design. I'm now seeing areas of different systems that are encapsulated incorrectly, at a higher level than class design. Recognizing the problem is the first step - and we're now working to rearrange and invert these models to more accurately reflect reality.

Pay attention to the pain that your application, infrastructure and other supporting services are causing you. You may be staring at evidence of a design problem, without realizing it.

Yesterday, I was involved two very separate yet very related conversations. One was via twitter with Colin Jack and Jimmy Bogard (which I was only a partial contributor to - mostly just reading their conversation) and another after work with a coworker. The short version of both conversations can be boiled down to encapsulation of logic surrounding collections that are held by entities. Rather than rehash all of the conversations, I wanted to specifically address a violation of encapsulation that I've seen many times when dealing with collections and business rules.

Let's look at a small example where we have a medical system that deals with Patients that are having Consultations with doctors. There is a need to keep track of the historic consultations and also the current consultation, if any. For simplicity we'll say that the current consultation is identified as being the most recent, based on a starting date, and that any previous consultation must have an ending date. A very simplistic implementation of an object model to represent patients and consultations may look something like this:

public class Patient

{

  private IList<Consultation> _consultations = new List<Consultation>();

  public IList<Consultation> Consultations { get { return _consultations; } }

  public Consultation CurrentConsultation { get; set; }

}

public class Consultation

{

  public DateTime StartingDate { get; set; }

  public DateTime EndingDate { get; set; }

}

Then, when the time comes to add a new consultation to the patient, making it the current one and closing a previous consultation, code may get called from somewhere in the application (like a code behind of a form, or if you're lucky, in the Presenter or Controller of an MVP/C setup), like this:

Consultation consultation = new Consultation{ StartingDate = DateTime.Now };

patient.Consultations.Add(consultation);

if (patient.CurrentConsultation != null)

  patient.CurrentConsultation.EndingDate = DateTime.Now;

patient.CurrentConsultation = consultation;

From a purely technical standpoint, there is nothing wrong with this code. It implements the business rule as defined. However, this code misses out on some great opportunities to encapsulate the rules we have into a process that can be called from anywhere that the system needs it - whether or not the code is in the specific presenter / controller that creates a new consultation or not. The very same code that comprises this implementation could easily be placed in the Patient object, abstracting the rules and process of creating a new consultation into a simple, single method call.

Before I show how I would approach that solution, though, there is one other implementation that I've seen recently that not only breaks encapsulation, but has to compensate for the lack of rules enforcement with logic in the wrong place. Instead of storing the current consultation as a set value, the CurrentConsultation property may have some logic in it to dynamically determine which consultation is the current one.

public class Patient

{

  private IList<Consultation> _consultations = new List<Consultation>();

  public IList<Consultation> Consultations { get { return _consultations; } }

  public Consultation CurrentConsultation 

  {

      get 

      {

          Consultation currentConsultation;

          DateTime mostRecent = DateTime.MinValue;

          foreach(Consultation consultation in _consultations)

          {

              if (consultation.StartingDate > mostRecent)

              {

                  mostRecent = consultation.StartingDate;

                  if (consultation.EndingDate == DateTime.MinValue)

                  {

                      currentConsultation = consultation;

                  }

              }

          }

      }

  }

}

This type of code is a huge encapsulation violation smell. Since our Patient object has no enforcement of the consultations that it holds, there is no way for us to really know which consultation is the current one. Because of this, the retrieval of the current consultation has to process the entire consultation collection and try to find the most recent consultation that has no ending date.

On top of the encapsulation issue, we have lost a great deal of performance. We now have to loop through the list every time we need the current consultation. If the list is small, this might not be such a bad problem, but as the list grows and as this code is used more and more, the performance problem may have a serious impact on the system.

Fortunately, the solution to the encapsulation violation, the enforcement of the business rules and the performance problem can all be wrapped up in to some very simple code. The first thing we want to do is prevent the ad-hoc addition of consultations to the patient. We still need to access the list of consultations, but we don't really have a need to modify it outside of the patient class itself. This can be done with a one-line code change to the Patient class's Consultations property:

public class Patient

{

  private IList<Consultation> _consultations = new List<Consultation>();

  public IEnumerable<Consultation> Consultations { get { return _consultations as IEnumerable; } }

  //ignoring other implementation details for the sake of illustrating the IEnumerable change

}

Now that we have prevented the ability to do ad-hoc consultation additions, we need a way to actually add consultations. While we are doing this, we also want to enforce the business rules of the current consultation as described earlier. This is where we are going to take much of the original code that we found in the presenter / controller and place it into the patient class directly.

public class Patient

{

  private IList<Consultation> _consultations = new List<Consultation>();

  public IEnumerable<Consultation> Consultations { get { return _consultations as IEnumerable; } }

  public Consultation CurrentConsultation { get; private set; }

  public void StartConsultation()

  {

      Consultation newConsultation = new Consultation{ StartingDate = DateTime.Now };

      if (CurrentConsultation != null)

          CurrentConsultation.EndingDate = DateTime.Now;

      CurrentConsultation = newConsultation;

      Consultations.Add(newConsultation);

  }

}

In the end, this code shows a better encapsulation of the business rules and logic that surrounds the need to maintain a list of consultations and a current consultation. With this code in place, we could simplify the presenter / controller that we talked about initially. Rather than being forced to know all of that logic in the presenter, we can make one simple method call:

 patient.StartConsultation();

With this one simple call, we have a guaranteed execution of the business rules that we need. This will allow us to recreate the ability to add a new consultation at any point in the application that we need, not just in the original presenter / controller that we were working with.

Side Note:

Part of the conversation via twitter revolved around where this type of logic should be encapsulated. From what I gathered, Colin tends to place this logic in custom collection objects, which would allow him to call patient.Consultations.Add() and still encapsulate the same business rules into that method. Like everything else in software development, there are multiple ways to solve the same problem. What does your specific situation, project, team, and business context need? Whatever your implementation needs are, though, we need to keep this type of logic and business rules enforcement well encapsulated in our systems.

A coworker recently asked if we should always abstract every object into an interface in order to fulfill the Dependency Inversion Principle (DIP). The question stunned me at first, honestly. I knew in my head that this was a bad idea - abstracting into interfaces for the sake of abstraction leads down the path of needless complexity. However, I wasn't able to clearly answer his question with specific examples of when you would not want to do this, at the time. I've been thinking about this for a few days now and I think I have a good, albeit very long winded, answer.

Before the question is answered, though, we need to step back and look at what DIP is all about. I've previously shown how to implement DIP and talked about why it's beneficial, so I won't be repeating that here. Rather, I want to talk about the language that describes DIP and what it really means.

Robert Martin's original definition of DIP is this:

A. High level modules should not depend upon low level modules. Both should depend upon abstractions.
B. Abstractions should not depend upon details. Details should depend upon abstractions.

The word 'abstraction' is used three times in the definition for DIP. So, in order to understand DIP, we have to first understand some of the basics of Abstraction.

Some Background On Abstraction

From Wikipedia (emphasis mine):

"In computer science, abstraction is a mechanism and practice to reduce and factor out details so that one can focus on a few concepts at a time"

I can't say it any better than this.

Abstraction can very directly lead to a system that is more understandable by helping us ignore the detail and implementation specifics, allowing us to focus on something at a higher level. This reduction in cognitive load can benefit someone that is reading the code by not forcing them to know the detail immediately. Additionally, abstraction is a form of encapsulation or information hiding, which again helps us to reduce cognitive load and produce better systems. From Wikipedia's entry on Information Hiding:

"In computer science, the principle of information hiding is the hiding of design decisions in a computer program that are most likely to change, thus protecting other parts of the program from change if the design decision is changed."

At the heart of abstraction and information hiding, we find the ability to change the system. The ability to change is an absolute requirement in software development and produces good design that is easier to work with, modify, and put back together as needed. The inability to change is directly called "bad design" by Robert Martin, in the DIP article.

Applying Abstraction And Encapsulation To DIP

So how does our knowledge of abstraction and information hiding play into DIP?

First and foremost, DIP never states that we should depend on explicit interfaces. Yes, in C# we have an explicit Interface as a form of abstraction. It is a separation of the implementation detail from the publicly available methods, properties, etc, of a class. Some languages, such as C++, don't have an explicit construct for interfaces, though. From Robert Martin's original DIP article, again:

"In C++ however, there is no separation between interface and implementation. Rather, in C++, the separation is between the definition of the class and the definition of its member functions."

A String As An Abstraction

Abstraction does always mean explicit interface constructs, as evidenced in C++. Nor does it always mean an abstract base class, which we also have available in .NET. In fact, languages such as Ruby don't really need either of these constructs. The duck-type nature of Ruby allows an implementation to be replaced at any point, without any special constructs. In .NET, though, there are a number of abstraction forms that we can rely on, explicitly. We have the obvious interfaces and base classes (abstract or not) - but we also have constructs like delegates and lambda expressions, and even the simple types that are built into the base class library.

Let's look at a simple string to illustrate abstraction. As I said in my SOLID presentation at ADNUG, we can invert our dependency on database connection information.  Rather than putting a connection string directly into our code that calls the database, we can use the string as our dependency and our abstraction. All we need to do is follow the basic DIP principle and provide the string as a parameter to the class that calls the database. We certainly don't need (or want, for that matter) to introduce a new interface or base class at this point. Our abstraction is simple enough to use a common type found in the .NET framework.

Other Forms Of Abstraction

Even if we are talking about an object, who says that the interface we are depending on has to be an explicit interface construct or base class? When I write a Domain Service that uses an Entity from my Domain, I don't create an explicit interface for that Entity. Rather, I use the Entity's inherent interface - it's public methods, properties, etc.

I also use delegates on a regular basis. By specifying my abstraction as a delegate, I can further decouple the depending object from the dependant code that it needs to call. I'd be willing to bet you have used delegates as abstractions as well. Have you ever created an event handler for something like a button click? There's a delegate's abstraction at work.

Abstract Judgement

The point is, there is not always a need to introduce an explicit interface or base class when inverting our dependencies. We still need to apply dependency inversion and provide our implementation as a constructor parameter (or setter, though I don't like setter injection). But, that dependency doesn't have to be anything more than the interface inherent to the object, or a simple type found in .NET.

You do have to be careful when making the call to not use an explicit abstraction with DIP, though. You can quickly turn your system into a ball of mud if you rely on a concrete class that is not intention revealing or well encapsulated to begin with. At the same time, too many abstractions can lead to needless complexity and make it very difficult to see the big picture of a system. Too few abstractions, though, will certainly lead to a rigid, immobile design that is hard to change. All of these problems are equally vicious - and I've been bitten by all of them in recent months.  It takes good judgement calls to determine when you do and do not need an explicit abstraction for a dependency. Unfortunately, good judgement comes from experience and experience comes from bad judgement. Don't be afraid to make bad decisions - make a decision, just be sure you can reverse that decision as easily as possible.

I had a lot of fun giving my SOLID Principles presentation at the Austin .NET User Group last night. It was a pleasure and an honor to be able to give back to the community that has supported me for so many years.

I'd like to thank everyone that came out to see the presentation, for being such a great audience. There were a lot of great questions and comments and good discussion. I sincerely hope that I've inspired at least a few people to continue digging into the SOLID principles.

As promised, I've posted my slides and the example code for download.

Additional resources for SOLID:

• Uncle Bob (Robert Martin)'s Principles Of Object Oriented Development
• Agile Principles, Patterns, and Practices in C# by Robert and Micah Martin
• Pablo's Topic Of The Month: SOLID

For the questions on legacy code from last night:

• Working Effectively with Legacy Code by Michael Feathers
• Jimmy Bogard's Legacy Code Presentation from Austin Code Camp 2008

If anyone has any questions about the SOLID principles, would like more information, etc, please feel free to contact me via the contact link on my blog(s) or via my email address listed at the end of the slides.

FYI - I'll be giving my S.O.L.I.D. Software Principles presentation at the Austin .NET User Group on Monday the 13th. This is the same presentation that I gave at Pablo's Day(s) of TDD last weekend, except I'll have the missing code in place and the slide errors fixed.

Here's the abstract that was posted via the ADNUG mailing list:

S.O.L.I.D. Software Development: Achieving Object Oriented Principles, One Step At A Time 

Almost every professional software developer understands the academic definitions of Coupling, Cohesion, and Encapsulation. However, many of us do not understand how to actually achieve Low Coupling, High Cohesion, and strong Encapsulation, as prescribed. Fortunately, there are a set of stepping stones that we can use to reach these end goals, giving us a clear cut path to software that is easier to read, easier to understand, and easier to change. This presentation will define not only the three object oriented goals, but also the five S.O.L.I.D. principle that lead us there, while walking through a sample application.

At some point this week or next, I will have my slides and code posted for people to download. (I know I promised this at PDoTDD. I got busy. Sorry). And as usual, there will likely be a group of people heading to Rudy's after the meeting, for extended discussions and snappy banter.

A few months ago, I posted some thoughts and questions on the proper use of Inversion of Control (IoC) containers and the Dependency Inversion (DI) principle. Since then, I've had the opportunity to do some additional study and teaching of DI, and I've had that light bulb moment for the proper use of an IoC container. I haven't talked about it or tried to present the info to my team(s) yet, because I had not verified my thoughts were on the right track - until recently. I got to spend a few hours at the Dovetail office with Chad, Jeremy, Josh, etc, and had the pleasure of being able to pick their brains on some of the questions and thoughts that I've had around DI and IoC. In the end, Chad confirmed some of my current thoughts and helped me put into a metaphor that I find to be very useful in understanding what Dependency Inversion really is - a cloud objects that can be strung together into the necessary hierarchy, at runtime.

Consider a set of classes that need to be instantiated into the correct hierarchy so that we can get the functionality needed. It's really easy to have the highest level class - the one that we really want to call method on - instantiate the class for next level down, and have that class instantiate it's next level down, and so-on, like this:

This creates the necessary hierarchy, but breaks the core object oriented principle of loose coupling. We would not be able to use ThisClass without bringing ThatClass along with it, and we would not be able to use ThatClass without bringing AnotherClass along with it.

By introducing a better abstraction for each class and putting Dependency Inversion into play, we can break the spaghetti mess apart and introduce the ability to use any of these individual classes without requiring the specific implementation of the dependent class.

For starters, let's introduce an interface for ThisClass to depend on and an interface for ThatClass to depend on.

Now that we have an interface that both of these classes can depend on, instead of the explicit implementation of the child object, we need to have the expected child object implement the interface in question. For example, we expect ThatClass to be used by ThisClass, so we will want ThatClass to implement the IDoSomething interface. By the same notion, we want AnotherClass to implement the IWhatever interface. This will allow us to provide AnotherClass as the dependency implementation for ThatClass. Our object model now looks like this:

What we have now is not just a set of classes that all depend on each other, but a "cloud" of objects with dependencies and interface implementations that will let us build the hierarchy we need, when we need it.

The real beauty of this is that we no longer have to care about the implementation specifics of IDoSomething from ThisClass. ThisClass can focus entirely on doing it's duty and calling into IDoSomething when it needs to. And, by passing in the dependency as an abstraction, we're able to replace the dependency implementation at any time - runtime, unit test time, etc. This also makes our system much easier to learn and understand, and most importantly - easier to change.

Now that we have our cloud of implementations and abstractions in place, we will need to reconstruct the hierarchy that we want so we can call into ThisClass and have it perform it's operations. Here's where Dependency Inversion meets up with Inversion of Control.

• To create ThisClass, we need an implementation of IDoSomething
• ThatClass implements IDoSomething, so we'll instantiate it before ThisClass
• ThatClass needs an instance of IWhatever
• AnotherClass implements IWhatever, so we'll instantiate it before ThatClass
• Once we have AnotherClass instantiated, we can pass it into ThatClass's constructor
• Once we have ThatClass instantiated, we can pass it into ThisClass's constructor

We end up with a hierarchy of objects that is instantiated in reverse order, like this:

We have now successfully inverted our system's construction - each implementation detail is created and passed into the the object that depends on it, re-creating our hierarchy from the bottom up. In the end, we have an instance of ThisClass that we can call into, with the same basic hierarchy of classes that we started with. The real difference is that now we can change this hierarchy at any time without worrying about breaking the functionality of the system.

Once we have our Dependency Inversion and Inversion of Control in place, we can start utilizing the existing IoC frameworks to automatically create our hierarchy of objects based on the advertised dependencies (an advertised dependency is a dependency that is specified as a constructor parameter of a class). Tools like StructureMap, Spring.net, Windsor, Ninject, and others, all provide automagic ways of creating each dependency of the object that is requested, all the way up/down the hierarchy. Utilizing one of these IoC containers can greatly simplify our code base and eliminate the many object instantiations that would start to liter our code. As I said in my previous post, I know all about what not to do with IoC containers. Good IoC usage, though, is another subject for another post.

I've seen this question a lot recently, and Scott Bellware asks it again in response to a post by Jimmy Bogard.

"Ok, but why are "contracts" important?  I can write concrete classes that interact with each other just fine without interfaces. And as Roy has already demonstrated, you don't need interfaces for test-focused SoC. To wit, any concrete class's signature is a contract with its user." -- Scott Bellware

First - a small psychology lesson in Cognitive Load theory and Chunking. A human brain can only hold so much information in it's short term memory. The "magic number" of 7 (+/- 2) has often been touted as the average number of concepts that a person can hold in short term memory, and still understand what's going on.

... and we need to recognize this in our code. If a developer who is reading the code has more than 7 (+/- 2) concepts, then that developer is not likely to understand the code they are reading. If you can't understand the code, you can't maintain the code. It's as simple as that. So why are contracts important? An intention-revealing interface, as a contract, can significantly reduce the cognitive load that is required to understand the code in question.

Abstraction and dependency inversion via interfaces help us achieve this understandability by letting a developer's mind chunk a process into what's really important - the "when" and "what" of the process, ignoring the "how". If you eliminated all abstractions and interfaces - even the interfaces that have only one implementation - you are telling a developer that they need to know the details of "how", not just the abstraction of "when" and "what". This puts an additional load on any persons' brain, and can quickly overload the person reading the code.

These theories go so far beyond just code, in software. When was the last time you saw a web site that had more than 8 or 9 buttons in it's navigation/menu, and you thought it was an intuitive and easy to use site? I'd be willing to bet that you thought the site was poorly organized and difficult to navigate. Interaction design is usually the first place that people apply cognitive load and chunking theories. Unfortunately, it's also usually the last. We need to break this cycle of overloading ourselves and our coworkers, and create proper abstractions in our code that fit easily within our own cognitive load, but more importantly in the cognitive load of other developers who have to read/maintain the code.

I was reviewing my calendar for the upcoming month of October, and it looks like I'm going to have a crazy-fun month!

October 3rd & 4th: Pablo's Day(s) of TDD

This is going to be a great couple of days of learning, and hopefully contributing, to spread 'the good news' of TDD.

October 7th: AgileAustin montly meeting

All about test automation! A subject I'm very interested in, recently.

October 13th: Austin .NET User Group (ADNUG)

Presentation by ... ME!

I'll be giving my SOLID Software Principles presentation, talking about why we want SOLID software, as well as how to achieve it.

October 30th - November 2nd: KaizenConf: Continuous Improvement in Software Development

Otherwise known as an Alt.NET Conference. This is going to be a great weekend of learning from the best and brightest minds in the Alt.NET community, as well as Tom and Mary Poppendieck!

... October is going to be a whirlwind of learning, for sure.

After some additional Twitter discussion with Jimmy last night, I realized that my previous response had neglected to distinguish between two very important contexts - new code and legacy code.

Considering a legacy system (as defined by Michael Feathers), I'm 100% on board with everything Jimmy is talking about and all of the techniques, troubles, and diminishing returns that he identifies.

I still maintain that 100% coverage should be the default result of true TDD when writing new code - and I apply this philosophy when writing new code inside of a legacy system. Write a test to prove you need the new code that you are writing... unfortunately, the distinction between new and legacy code does get blurry and can be difficult to navigate when working in a legacy system.

For more info on legacy code, in this context, and how to deal with it:

• Working Effectively With Legacy Code - the book by Michael Feathers
• Working Effectively With Legacy Code - Article on ObjectMentor by Michael Feathers (consider it the cheat-sheet for the book)
• Jimmy Bogard - He's done a "Legacy Code" presentation several times, based on his own use of the Michael Feathers techniques.

I've talked about code coverage and software quality with a number of people on Twitter in the last month or so, including Jimmy Bogard, who has a nicely written post about Quality and code coverage.

It's no secret that I have some fairly radical opinions on code coverage and that I don't agree with Jimmy in these regards. I firmly believe that 100% code coverage, in whatever form we can get, is a reasonable goal and not past the point of diminishing returns. As such, I'd like to respond to some of what Jimmy is saying and expand on it with my own thoughts.

But first - I don't like to draw hard lines between "Unit" tests and "Integration" tests - that just blurs the problem even more because we can argue that you are covering code or not, in "unit" tests all day long, based on this blurry line. This is not the discussion I want to have, so I am lumping "Unit" and "Integration" tests into "Unit" tests as a whole.

Secondly, I'd like to say that code coverage is not always an indication of quality - in poorly written systems, it's merely a measure of code coverage. However, quality code is another subject of much debate.

On to the response!

.............................................

"The idea is that the team, all practicing TDD, should dutifully measure and add unit tests until they reach the assumed pinnacle of unit testing: 100% coverage."

There are several assumptions and statements in this that I don't believe are true.

For one, TDD and Unit Testing are not the same - no where near the same. Unit Testing lets you add tests as you see fit - after you write the code, while you write the code, whenever you want. You can even do test-first unit testing. True TDD, however, is a pure pragmatic approach to software development in that you never write code that does not have proof of need to exist - a test that says it needs to be there. 

The "pinnacle" of TDD and Unit Testing are not the same thing. Unit Testing - the act of adding unit tests when you see the need - may have a goal of 100% coverage. TDD, on the other hand, has a goal of only implementing the code that has been specified as needed. 100% coverage is the default in TDD because you never write code you don't need.

If you are going through the "measure and add unit tests" process, I would put money down to say that you are doing Unit Testing and very little TDD. The process should be something more like "measure to make sure we haven't added anything we don't need yet". 100% coverage is a side effect of TDD.

"The general motivation behind 100% coverage is that 100% coverage equals zero bugs."

Unfortunately, this is the motivation for many people who write unit tests - but it's the wrong motivation. You will never be 100% bug free just because you have 100% code coverage. There will always be some business case or external system factors that cause a bug now and then. That doesn't mean we should be ok with letting bugs into our system. On the contrary, one of our motivations (not the only one) should be to prevent as many bugs as possible from getting into the system.

"NCover is a powerful tool, but it still doesn't support all types of coverage.  Attaining 100% coverage in NCover still means there are paths that we haven't tested yet, which means there are still potential bugs in our code. "

I think the logical conclusion of this argument is that if we don't have a tool that supports true 100% coverage, then we shouldn't use that tool at all. I know I'm inserting my own conclusions into Jimmy's words - that's simply the conclusion that I came to from these statements. And this is a very bad conclusion. Do you walk around naked simply because wearing clothes would get them dirty and you'd have to wash them? Both of these are akin to the broken window syndrome. Use the tools that are available to the extent that they can operate, and find better tools when you can.

"If 100% coverage is a goal,"

100% coverage should NEVER be a goal - it's merely an indication that you are on the right path and working pragmatically.

"In recent projects where we measured coverage several months in to the project, we saw regular numbers of 90% coverage.  This was on a team doing 100% TDD." ... "So are we doing TDD wrong?"

So, are you doing TDD wrong? Yes. You are unit testing in the guise of Test-First-Development and not adhering to the spirit or pragmatism that TDD wants.

"Every test introduced covered behavior we considered interesting.  If behavior isn't interesting, we don't care about it. "

If you have code that is not interesting and you don't care about it - why is it in the system? If you don't care about the code, then it makes no difference whether or not that code works correctly. I would be appalled to hear that this is true. If it is true, then that code should not be in the system to begin with.

"If tests are a description of the behavior of the system, why fill it with all the boring, trivial parts?  The effort required to cover triviality is just too high compared to other ways we can increase value."

I think your definition of "behavior" is wrong. From wordnet.princeton.edu/perl/webwn:

Behavior: "the action or reaction of something (as a machine or substance) under specified circumstances;"

So, how is throwing an exception when a parameter is null, not behavior? If you are doing defensive programming to make sure you have all the data you need, then you are creating behavior - system behavior, not business process behavior. Behavior is still behavior, though. The only possibility of code not being behavior is a simple data point. If you have a data point in your code that is not covered by a unit test of behavior, then you don't need that data point. Yes, yes, I know - "integration", "nhibernate", "not my data source", "it has to be there for ...". If you argue that you need a property for some external portion of the system to operate correctly, then you should be proving that you need the property by covering it with a unit test that specifically says what it is for and shows how it is used. If you don't, then there's no way of knowing why the property exists and it may be deleted because it looks like nothing in the system uses it. Or, possibly worse, you end up with a lot of dead code in your system because you are afraid of deleting anything.

"Missing in the 100% coverage conversation is the effort required to get to 100%.  Attempting to get another 5% takes equal effort of the previous 10%.  The next 2% takes equal effort of the previous 15%, and so on."

I'd like to know where you got the math. If you are talking about unit testing - even test-first unit testing - i can understand the pain you're talking about, here. Why should I bother setting up another test fixture and getting all the state of my objects in place for a simple null reference check? blech - that totally sucks, is boring, is tedious, etc.

However, my typical process of achieving 100% is to only write code that is needed for the system to work, by specifying how the system will work through tests and then coding it. If I have a null reference check somewhere in my code it should be because I already have a test that specifies why it's needed.

"This is called the law of diminishing returns.  As we get closer and closer to 100%, it takes vastly more effort to get there.  At some point, you have to ask yourselves, is there value in this effort? "

Here's the real crux of the problem - you are assuming that you started with, or at some point has less than 100% coverage. It takes zero additional effort to stay at 100% coverage if you start at 100% coverage and maintain it through pragmatic TDD.

"Often, bending code to get 100% can decrease design quality, as you're now twisting the original intent solely for coverage concerns, not usability, readability or other concerns."

If you find yourself bending code to get 100% coverage, you have one of two situations (if not both): 1) poor design in your code, 2) poorly written tests. I would bet that you have both because they are a vicious circle.

"Measuring coverage is an interesting data point, as are other measures such as static analysis.  But in the end, it's only a measure, an indication. "

100% agreed!

"It's still up to the team to decide on the value of addressing missing areas,"

This implies that you don't have 100% coverage, already. In this situation, 100% agreed. I'm in this situation right now - it's hard to know what tests create the most value, at this point. However, any new code - even code changes to existing code - are done TDD style. So, while we don't yet have 100% coverage, we are increasing coverage all the time by never writing code without a test (that's part of the goal, anyway... no one is perfect, and it takes serious discipline to do this)

"with the full knowledge that they are still limited to what the tool measures."

Agreed, again. If you don't know the limitations of the tools you are using, you're in trouble.

.............................................

The boilerplate argument that I am making is that you should never write code that is not required by a customer / consumer. However, as I've Previously Talked About, there's a high likelihood that you have not identified all of your customers / consumers. Take NHibernate, for example. I have a project that needs about 30 different properties for a specific NHibernate query to be executed. The NHibernate query is the only place they are ever used - no code reads them, only a search screen writes to them, then the NHibernate query loads data from the table that they are mapped to. When I first wrote this code, my coverage was at around 30%. This situation is one of many that I've been in recently, that has lead me to believe that other code must be considered a customer / consumer of your code. If I deleted any of these properties, then the NHibernate query would fail. Therefore, the properties are valuable to NHiberate. Since these properties are valuable to at least one of the consumers of my code, I should prove that they need to exist by writing a test that proves they are needed and why.

I gave a presentation to my team on S.O.L.I.D. software development principles, talking about how these five principles enable us to achieve High Cohesion, Low Coupling, and proper Encapsulation in our software projects. If you're unfamiliar with the SOLID principles, they are:

• SRP: Single Responsibility - One reason to exist, one reason to change
• OCP: Open Closed Principle - Open for extension, closed for modification
• LSP: Liskov Substitution Principle - An object should be semantically replaceable for it's base class/interface
• ISP: Interface Segregation Principle - Don't force a client to depend on an interface it doesn't need to know about
• DIP: Dependency Inversion Principle - Depend on abstractions, not concrete detail or implementations

You can find a lot of good info on SOLID via Robert Martin (the man who coined the acronym) and the Los Techies crew:

• http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
• http://www.lostechies.com/blogs/chad_myers/archive/2008/03/07/pablo-s-topic-of-the-month-march-solid-principles.aspx 

The final slide in my presentation shows the before and after of migrating a small app that reads a file and sends an email, from being 100% coded in the Winforms code, to a SOLID code structure.

During the summary / review, I related the five SOLID principles to High Cohesion, Low Coupling, and Encapsulation through these descriptions:

Low Coupling:

By abstracting many of our implementation needs into various interfaces and introducing the concepts on OCP and DIP, we’ve created a system that has very low coupling. Many of these individual pieces can be taken out of this system with little to no spaghetti mess trailing after it. Separating the various concerns into the various object implementations has also helped us ensure that we can change the system’s behavior as needed, with very little modification to the overall system – just update the one piece that contains the behavior in question.

High Cohesion:

This really is a direct result of low coupling and SRP – we have a lot of small pieces that can be stacked together like building blocks to create something larger and more complex. Any of these individual pieces may not represent much functionality or behavior, but then, an individual piece isn’t much fun to use without a bunch of other pieces. DIP has also allowed us to tie the various blocks together by depending on an abstraction and allowing that abstraction to be fulfilled with different implementations, creating a system that is much greater than the mere sum of it’s parts.

Encapsulation:

True encapsulation is not just making fields private and hiding data from external objects – but hiding implementation details from other objects, depending only on the abstractions and expected behaviors of those abstractions. LSP, DI, and SRP all work hand in hand to create true encapsulation in the new project structure. We’ve encapsulated our behavioral implementations in many individual objects, preventing them from leaking into each other – and we’ve ensured that the dependency on those behaviors is encapsulated behind a known interface. We’ve hidden the implementation details and allowed for any implementation to be put in place for that interface definition through DIP. At the same time, we’ve done the necessary due-diligence to ensure that we are not violating any of the individual abstraction’s semantics or purpose (LSP), ensuring that we can properly replace the implementation as needed.

The end result of our effort has created a system that appears to be complex on the surface – after all, there’s now 13 blocks in our diagram compared to the original 2. However, the apparent complexity of this system is diminished by the simplicity of each individual interface and object. Many small, independent, simple pieces have been wired together to create a larger overall system behavior that can be described as complex. Yet any of these implementations can be changed and/or replaced – very easily.

At some point, I'm hoping to post the entire slide deck and presentation / code, but for now I thought the summary that I sent to the team was worth sharing with the rest of the world.

Earlier today, I had a conversation with a coworker concerning dependency container, dependency injection frameworks, and the root dependency inversion principle. My advice in the end, was to completely avoid the use of DI tools until the team as a whole understands the cost, benefits, and potential pains of manual dependency injection (pain being relative, and usually a sign of a learning opportunity). Part of the conversation also revolved around what constitutes the complete overuse and abuse of DI or IoC tool - which I can easily speak about due to extensive personal, self-inflicted, love-affair-of-IoC induced pain over the last year. However, the one thing I could not speak to was the correct use of a DI / IoC tool, because I believe that I have never used one correctly - or at least, my limited experience in using one correctly is so limited, I can't seem to separate it from the incorrect use.

I’ve heard other developers (Jeremy miller, jimmy bogard, and others) say that they want to see as little of a dependency container / injection tool in their code as possible. This gets to the heart of what I was trying to convey earlier, but I’ve never had a good understanding of where you would actually allow a dependency tool to be used under those guidelines.

On the way home from work tonight, I had (what I think is) a small epiphany around the idea that the dependency container should be limited to two key areas: the various implementation specifics (UI, database, etc) and the application layer. I think the use in the implementation specifics, for whatever use they're needed, is valid since these implementations are not unit tested to begin with. But I would highly recommend limiting the container’s use in these scenarios, for the same over used, abused reasons that I'm so intimate with already.

More appropriately, the application layer seems to be the appropriate location to resolve the dependencies by using the DI tool to instantiate the object that needs the dependencies, automatically resolving the dependencies for us – not requesting the dependency directly.

The best example I can think of, off-hand, is a workflow coordination service in the application layer. Let’s say your workflow moves from FormA to FormB in a windows app. The workflow class would use the DI tool to instantiate the ProcessAPresenter, which would resolve the registered IProcessAView as a constructor dependency. Then when this form is done, the workflow coordination class would use the DI tool again to instantiate the ProcessBPresenter and resolve the IProcessBView (and ISecurityService and whatever else) constructor dependencies.

The key here is that we are allowing the application layer to use the DI tool, and not the other way around – not letting the DI tool instantiate the application layer - and not using the tool as a simple IoC container to resolve dependencies internally from the object that needs the dependency.

These are primitive, unverified thoughts at this point, and need to be taken as such. I think this is a good start for a correct use of a DI tool, though. Additional implementation experience within this model would help to expose additional constraints and allowed uses, I would imagine.

How are you using your DI / IoC tools? What are your thoughts on the subject? Your pains, your joys, your sorrows? And from a purely selfish perspective - am I on the right track, here?

In addition to our 'Code Review Challenge' that I discussed recently, we have been using another retrospective game - 'Name That Standard'.

The idea is fairly simple. At the beginning of any retrospective, we go around the room and ask each team member to list out a standard that we are using in our project. This can be any standard that anyone feels the team is using or needs to be using, including coding conventions, architectural patterns, business and project management processes, communication means, etc. The intention is similar to the intent of 'The Code Review Challenge' - socialization of the the system. Only in this case, we are applying the term 'the system' at a much higher level - our organizational and project management / implementation system as a whole.

Some thoughts on running 'Name That Standard':

• Require not only naming of the standard, but accurate description of the standard and an example of where it is used. Alternatively, you may want to list a brief description out on a projector or whiteboard and see who can name the standard being described.
  
• Encourage the team to think about the team as a whole, not just their area of responsibility.
  
• Encourage the team to list standards that they don't understand, so that the rest of the team can chime in with assistance and help the team member in question grow
  
• Don't just let people yell out standards. go around the room, one person at a time. we got a little chaotic at first, and it was hard to know who said what.
  
• Encourage people to ask questions like 'i thought we were using this standard, but i saw someone else doing that standard'
  
• Encourage the standards to become more and more low level and detailed, over time (over multiple iterations)
  
• Encourage the standards to evolve and change for the better, over time (over multiple iterations)

In our implementations of this game, we've managed to get a fairly good list of standards written down and also resolved some questions on what standards we actually are using. For example, we had 3 dynamic mocking frameworks in our code base - Rhino Mocks, Moq, and a home grown one that we lovingly called 'BrandoMocks' (named for our team member, Brandon, who wrote it for fun one night). The end result of that discussion was to throw out Moq and BrandoMocks, adopting Rhino Mocks as our standard for this project.

Having done 'Name That Standard' in two consecutive iteration retrospectives, I think I can say that it has been very successful. The team as a whole is becoming more and more aware of the standards that we are using, and the individual members are beginning to contribute more and more to the standards.

With each passing iteration, and each new technique for creating collective ownership and socializing the system, I see the team coming together and really forming a team vs. a bunch of individual developers working on the same system.

Test Driven Development Is To Unit Testing As Interaction Design (IxD) Is To Accidental Design

One of the major problems with writing unit tests after the code is that it is very natural to write tests that prove the code works the way it was written, instead of the way it should work. Writing code test-first (via whatever flavor of Test Driven Development) flips that scenario right side up. TDD helps to ensure that code is written to work according to specifications and not the other way around.

Similarly, designing and/or implementing a UI after a behavior or process has been coded is a likely to result in a UI that fits the code model and not a model that fits the needed interaction and workflow. This situation must also be flipped right side up - interaction design should be done before, or at least in parallel, with coding. It cannot be left to accidental or incidental happenstance - interaction design must occur with proper interaction patterns and practices in mind.

Overlapping IxD and TDD

To overlap interaction design and test driven development, there are a few key words that need to be borrowed from interaction design. Fortunately, they easily fit within TDD development techniques and philosophies.

Epistemic work is exploratory in nature, or a process of trial and error through research. TDD and interaction design sketching are both epistemic. TDD explores API possibilities and allows easy trial and error to find the simplest implementation for what you need at the time. UI design sketches also allow you to quickly explore interaction designs - whether it's a white board, pencil and paper, graphics design software, or even quick-hack forms layout in IDEs. You can quickly and easily throw away a bad API in TDD and you can quickly and easily throw away a bad UI/Interaction design when you have nothing more than pencil sketches or white board drawings.

Pragmatic work is very structured and step-by-step in it's nature, implementing patterns and practices to fulfill what is needed at the moment. Implementing Code after writing unit test and implementing UI after designing around constraints are both pragmatic. TDD is pragmatic in that you only implement what is needed to properly pass the tests that have already been written. Similarly, with previously designed interactions and UI elements, implementation can be easily limited to what is needed for the UI.

With epistemic and pragmatic work covering both interaction design and test driven development, it seems that they are a natural pair. An analysis of a user story and it's acceptance criteria will create the unit tests that we need. At the same time, the same analysis can be applied to interaction designs. Additionally, a strong understanding of how a UI will look can have a profound impact on the code that is written, and vice-versa. Therefore, it is natural that interaction design and test driven development are done at just-in-time intervals - before the real work is implemented in the real code and UI platform.

IxD As Part of "Done"

Despite interaction design being a required part of UI development, not all user stories require a UI. Interaction design may not fit into a swim lane board or be part of every story's "done" criteria. However, interaction design will always be done for any story that does have a UI - it may simply be an accidental or incidental part of the software development process for a team, though. If it's safe to assume that the work will be done, then it is the team's responsibility to ensure that it is done correctly. Don't let interaction design happen accidentally or incidentally in the development process. Set a standard of always including interaction design in the development process, the same way Model View Presenter/Controller is a part of development.

Earlier this week, I was doing a training session with my current team, building a sample Org Chart application. One of the user stories we were working with, talked about assigning managers to department and had the following acceptance criteria:

• When assigning a manager to a department, ensure that they are an employee of that department, and remove them from their old department

The basic code that we came up with worked well and was encapsulated into a service object in the domain.

public class AssignmentService

{

    private IDepartmentRepository _repository;

    public AssignmentService(IDepartmentRepository repository)

    {

        _repository = repository;

    }

    public void AssignManagerToDepartment(Employee manager, Department assignedDepartment)

    {

        Department previouslyAssignedDepartment = _repository.GetDepartmentForEmployee(employee);

        if (previouslyAssignedDepartment != assignedDepartment)

        {

            previouslyAssignedDepartment.RemoveEmployee(manager);

            _repository.Save(previouslyAssignedDepartment);

        }

        assignedDepartment.AssignEmployee(manager);

        assignedDepartment.AssignManager(manager);

        _repository.Save(assignedDepartment);

    }

}

Problems

At first glance, this code seems simple enough, but we quickly ran into a problem when we started coding for another story and acceptance criteria.

• When creating a department, allow a manager to be assigned

Off-hand, this seems like a very simple situation. We'll just call the AssignmentService object after we create the Department. However, there's a large potential problem with this - the AssignmentService handles the entity persistence for both of the departments that were changed, when the manager is assigned. With that in mind, the developer working on the new requirement would have to ensure that the department is created with all of it's required fields and information before calling this service object. If the developer allows the manager to be assigned before the department's info has been completely filled in, we may accidentally allow an invalid state for a department object when calling the AssignmentService. Since the AssignmentService tries to save the department being assigned, we may also have created an semantic coupling between the department creation process and the AssignmentService - the developer needs to know that the AssignmentService will save the department. If they don't know this, they may try to save the same entities more than once, may try to save an invalid state, etc, etc, etc.

Unfortunately, we didn't have time to fix the problem during the training session - we had to stop the training almost immediately after we discovered this issue. I've got a few ideas running around in my head, on how to fix this, but I'm not 100% sure that I like any of them.

Possible Solutions

First and foremost, though, it's becoming rather apparent to me that we want to avoid entity persistence calls from within the domain service objects. This essentially means that the AssignmentService object would have to return the assigned department, and the 'previously assigned department' objects, so that the coordinating presenter could save them both. That seems a little odd, if you ask me.

One of my coworkers suggested that we allow the service to save the previously assigned department, since that is an self-contained operation in the method (the method loads the department, modifies it, and saves it) and then return the assigned department so that the presenter can save it. This option seems a bit better off-hand, but I'm still worried about the possible side effects of the AssignmentService saving the previously assigned department.

Questions

What are your thoughts on this situation... How would you solve the dilemmas that I'm pointing out? Is there a better design for the model that would eliminate this problem and make the department persistence more obvious, without having to 'know' when it is persisted?

Is the idea of not allowing entity persistence from a domain service object valid? Should we go with allowing the self-contained operations to persist the entity in that operation?

Or ???

Over the weekend, I spent some time refactoring some business rules in my current project, to use the ISpecification<t> framework that I previously created.

One of my coworkers had asked about logical groupings - making sure that we can handle complex logic such as, "(this and this) or this". Since my project actually made use of logical groupings I wanted to share with the world and show how easy it is.

In this example, I have a Date Range that is being selected – a From date and a To date. The rules are as follows for validating the range:

• If a From date and a To date are provided, the From date must be equal to, or before, the To date
• Or, you can specify a From date without To date
• Or, you can specify a To date without From date

The following is the actual code that I’m using to define these business rules, via the ISpecification<t> framework. 

PredicateSpec<ExtractSearchCriteria> fromDateProvided = new PredicateSpec<ExtractSearchCriteria>(crit => crit.FromDate > DateTime.MinValue);

PredicateSpec<ExtractSearchCriteria> toDateProvided = new PredicateSpec<ExtractSearchCriteria>(crit => crit.ToDate > DateTime.MinValue);

PredicateSpec<ExtractSearchCriteria> fromDateBeforeToDate = new PredicateSpec<ExtractSearchCriteria>(crit => crit.FromDate <= crit.ToDate);  

ISpecification<ExtractSearchCriteria> validDateRangeSpec =

    (

        (fromDateProvided.And(toDateProvided))

        .And(fromDateBeforeToDate)

    )

    .Or(

        fromDateProvided.Not(toDateProvided)

    )

    .Or(

        toDateProvided.Not(fromDateProvided)

    );

Here, I am defining the core of the business rules – to check if the From date was provided, to check if the To date was provided, and to check if the From date is before or equal to the To date. From these core specifications, I can build the aggregate spec that handles all of the stated rules for validating the date range.

There are three logical groupings using parenthesis to represent each of the business rules that can comprise a valid date range. The first logical grouping checks to see if both a From and To date are provided, and checks if the From date is before or equal to the To date, if both are provided.

(

    (fromDateProvided.And(toDateProvided))

    .And(fromDateBeforeToDate)

) 

The second group is specified with an Or, and checks to see if we provided a From date but not a To date.

.Or(

    fromDateProvided.Not(toDateProvided)

)

And the third group also uses an Or, and checks to see if we provided a To date but not a From date.

.Or(

    toDateProvided.Not(fromDateProvided)

)

The end result is that we have one of three possible ways for this date range to be valid – all logically grouped and relatively easy to read (compared to a bunch of if-then statements, at least). Even better – we re-used two out of three simple ISpecification<t> objects to create all of the rules for this validation check. This helps to create a validation that is not only easier to read, but easier to maintain and enhance in the future.

My Previous Post talked about a sample Specification pattern implementation. Today, during a discussion with some team members, I had the revelation of how fluent interfaces are really built and how they operate under Closure of Operations. So I decided to expand my previous sample code with a more operators on the spec class, and I came up with a very powerful fluent interface for creating specifications.

Here is the example I coded into my spike, to illustrate a fluent interface. Note that I changed nothing in my implementation of the ISpecification<t> or base Specification<t> from yesterday - other than to add the "Or" and "Not" specifications. The fluent interface was already there and available!

//show the real power of Closure Of Operations, in creating a fluent interface!

ISpecification<Foo> chainedSpec = equalSpec

    .And(passingSpec.And(trueSpec))

    .Or(passingNotSpec)

    .Not(failingSpec.Or(falseSpec))

    .And

    (

        passingSpec.Or

        (

            failingSpec.Not(falseSpec)

        )

    );

I'll leave it to you, the reader, to implement the "Or" and "Not" specification classes. It should be pretty simple, based on the code I posted yesterday.

I can see how we could easily use an a "Closed" interface like this, to create a more fluent implementation of business rules and logic. I've often implemented multiple if-then statements in order to evaluate equality and equivalence, via multiple methods and a lot of ugly code. This specification implementation may be the answer to that ugliness.

And suddenly, Ayende's UnitOfWork and NHibernate's ICriteria make a lot more sense to me... I love those "AHA!" moments.

Inspired by my recent readings in Domain Driven Design - specifically Chapter 10, "Supple Design" - and recent posts by David Laribee and Nigel Sampson, in combination with the recent pains I've been putting myself through, trying to test query generation code in a search screen, I decided to spike out a quick example of a reusable Specification implementation.

Rather than repeat what's already been said, I'm just going to get straight to the code.

using System;

namespace Spec_Spike

{

    class Program

    {

        static void Main()

        {

            Foo foo1 = new Foo();

            foo1.Bar = "Test";

            ISpecification<Foo> equalSpec = new Specification<Foo>(foo => foo.Bar == "Test");

            ISpecification<Foo> notEqualSpec = new Specification<Foo>(foo => foo.Bar != "Not Equal To This Text");

            ISpecification<Foo> falseSpec = new Specification<Foo>(foo => false);

            ISpecification<Foo> passingSpec = equalSpec.And(notEqualSpec);

            ISpecification<Foo> failingSpec = passingSpec.And(falseSpec);

            Console.WriteLine(equalSpec.IsSatisfiedBy(foo1));

            Console.WriteLine(notEqualSpec.IsSatisfiedBy(foo1));

            Console.WriteLine(passingSpec.IsSatisfiedBy(foo1));

            Console.WriteLine(failingSpec.IsSatisfiedBy(foo1));

        }

    }

    public class Foo

    {

        public string Bar;

    }

    public interface ISpecification<t>

    {

        bool IsSatisfiedBy(t obj);

        ISpecification<t> And(ISpecification<t> lhs);

    }

    public class Specification<t>: ISpecification<t>

    {

        private readonly Predicate<t> _pred;

        public Specification(Predicate<t> pred)

        {

            _pred = pred;

        }

        protected Specification(){}

        public virtual bool IsSatisfiedBy(t obj)

        {

            return _pred(obj);

        }

        public ISpecification<t> And(ISpecification<t> andSpec)

        {

            return new AndSpecification<t>(this, andSpec);

        }

    }

    public class AndSpecification<t>: Specification<t>

    {

        private readonly ISpecification<t> _spec1;

        private readonly ISpecification<t> _spec;

        public AndSpecification(ISpecification<t> spec1, ISpecification<t> spec)

        {

            _spec1 = spec1;

            _spec = spec;

        }

        public override bool IsSatisfiedBy(t obj)

        {

            return (_spec.IsSatisfiedBy(obj) && _spec1.IsSatisfiedBy(obj));

        }

    }

}

Drop this code into a console app in C# 3.5 and watch the magic happen. Here's the output:

These are the results that I expected - the first 2 individual specs passed, the first combined spec passed, and the last combined spec failed.

Overall, I'm fairly excited about the possibilities here. I'm thinking that I may actually be able to properly unit test the query generating code in my search screen with this basic technique. I'm still not 100% sure on that, but I plan on trying, anyway.

For more information on the Specification pattern, I highly recommend you read the previously linked posts by David Laribee and Nigel Sampson, in addition to reading all of the Domain Driven Design Book. This is one of those books that should fundamentally change the way you think about software development.

A coworker asked this question about our automated integration test suite, recently:

"with the RowTest feature of NUnit, does the test data being used need to be hard coded into my code or can it be a variable that gets defined based on some rule?"

Here is my response, outlining the need to know the state of the database (know every value of every field, in every table) before and after the test suite is run:

The short answer is that the tests will have data hard coded into them.

This may be a change from how you have looked at integration testing, previously – where the data is changed on each run, so as not to duplicate the data in the system. Rather than changing the data that is used during each test, we need to ensure that the data is exactly what we expect, before and after each test. If the data is not what we expect before the test is run, the test cannot produce the data that we expect to have after it is run. If the data is not what we expect after a test is run, the test fails.

At a very high level, automated tests against the database require three things:

1. A known, state of ALL data in the database
2. A known, suite of unit tests that manipulate said data in predictable ways
3. An expected, verifiable suite of data returned by the software and manipulated in the database

The data is not necessarily "perfect" in that it has no flaws in it – rather, every last character in every last field of every last table is known, and all of the ramifications of that data is known with the explicit purpose of that data being there to support the automated tests. The automated tests expect this "perfect" data to produce the desired results and verify that the system works as expected. Any data that is manipulated in the system is expected to be in a predictable state, so the tests can verify that the system manipulated the data correctly.

When it comes to implementation of this, there are some fairly strict requirements for it to work:

1. Before the first test is run from the test suite, the data must be exactly what the tests expect
2. After the tests have run, the data must be exactly what the tests expect
3. Repeat for each run of the test suite

What this really means is that we cannot add or modify data between two test runs, without changing the tests that work with the data. In order for the tests to run multiple times throughout the day and/or on-demand, we have to ensure that the data being manipulated is what we expect it to be. Therefore, the first step of any run of the test suite is to revert the database to the known state, through sql scripts or code that are automatically run before the tests are run.

I've been toying around with various ideas in code, and I've come to the conclusion that the following formula is true:

Lambda Expressions + Func<> and Action<> = Easy Command Patterns

As a very contrived, quick example:

public class DoStuff

{

    private Func<CommandResult> Step1 { get; set; }

    private Func<CommandResult> Step2 { get; set; }

    public DoStuff(Func<CommandResult> step1, Func<CommandResult> step2)

    {

        Step1 = step1;

        Step2 = step2;

    }

    public void DoTheStuffProcessing()

    {

        CommandResult result = step1();

        if (result.Succeeded)

        {

            step2();

        }

    }

}

It's ... so ... beautiful ...

There are a few examples in The Toyota Way, of American manufacturing companies that were thought to be world class Lean Manufacturers. When representatives from Toyota began working with these companies, though, the result was quite a shake-up. The companies that were thought to be lean, were merely implementing some of the surface level processes of lean manufacturing. They had not realized that there was a root cause of the process that they were implementing.

The Principles, Not The Process

A coworker recently asked me to help out with a presentation on Model-View-Presenter. He had some specific questions about why we would want to use MVP vs. MVC vs. any other UI pattern. Many of the questions centered around various benefits that have been promoted by myself and others out in the developer world via blogs and articles - items such as testability, changeability, flexibility, and other "ilities". My answers to some of these questions surprised me.

Model-View-Presenter is often thought of as a journey and an end in itself. When I first started learning MVP just over a year ago, this is exactly how I saw it. I started with the intention of learning how to separate my core process from the form implementation, with the intention of being able to unit test the core process. I was working under the impression that I could find a way to implement MVP that would truly allow me to swap out my UI - to the point of changing the workflow within the UI. I had the goal in the mind, of being able to write one presenter and have it support WinForms, WebForms, Web Services, and any other client-facing API.

One year and several projects later, I have some good lessons learned. I have implemented various forms of MVP - automagic view injection, manual everything, and a lot of ideas in between. My understanding of MVP is fairly solid, and I am capable of implementing it in WinForms, WebForms, Web Services, and other client-facing API's.

What I have realized recently, is that MVP is not (or, should not be) a journey and a goal in itself. MVP, MVC, and other UI related patterns are actually an effect of other underlying principles. The underlying principles of object oriented development - such as Separation of Concerns, Single Responsibility Principle, Dependency Inversion and others - are the real cause of MVP, MVC, etc. Testability, flexibility, and the other "ilities" are merely side-effects of good design and implementation. And while Testability is a valid goal in itself, seeking it as the primary or only goal will only lead you part way down the path. You will end up like the American manufacturers that thought they were world class lean companies, only to find out that they had barely scratched the surface of lean process.

The Journey, Not The Goal

Despite the success of Toyota as the creators of lean manufacturing, they believe that they are continuously learning about lean, so they can continuously improve. Similarly, despite my many years of software development experience, I now believe that I know little about software development. I have a good knowledge of .NET and general knowledge of Object Oriented Development, but I don't know much about other paradigms like Functional Languages or Duck-Typed languages such as Ruby.

In my professional growth as a developer, I have set goals for myself at various times. Most of these goals were set with the belief that they would make me a great developer. However, I've found that every time I have reached a goal, there is an entirely new world of possibilities ahead and what I thought was a great developer was merely the beginning of a journey. I've discovered that the journey itself is often far more important than what I believed the goal to be.

I'm not advocating that you forget about setting goals and let your career happen as it will - that's a great way to become burned-out, outdated, and irrelevant. What I am advocating is that when we have a goal in mind, we set out with the understanding that the goal itself may only be a small leg on a long journey. In this case, the journey is the process of learning. The journey itself should be viewed as the long term strategy, to the point where you allow the individual goals to come and go as needed. You may not accomplish a specific set goal - but the experience gained while working toward that goal may open up opportunity for other goals to be reached easily.

Learn The Principles Via The Process

All this talk about knowing the underlying principles is great - but how can we understand the principles and philosophies without knowing the process? Unfortunately, I don't know if that's possible for most of us. I'm sure that somewhere out there, someone has read SRP, DI, SoC, and other principles, and as a result began writing code that was modeled in a manner similar to MVP (Martin Fowler, for example). For the majority of us, though, learning the principles is done by first learning the process - study the implementation of the effect to learn the cause.

Conclusions

The journey itself is how we build our experience and understanding. For most of us, that journey begins with a goal of learning a new process. Find a goal or two that you believe will help you be a better developer and learn the processes. Along the journey of reaching for this goal, always keep in mind that there is an underlying principle or philosophy that enables the process. By learning the principles and philosophies behind the process, the process itself becomes a trivial matter - an implementation of the principles and philosophies, and not a goal in itself.

The Toyota Way mentions the concept of Jidoka in chapter 1 (and probably other places that I haven't read yet). On page 6, in the "4 P" diagram, jidoka is described as

"Stop when there is a quality problem"

Wikipedia calls the same concept "Autonomation" and says it may be described as

"intelligent automation" or "automation with a human touch"

and

"At Toyota this usually means that if an abnormal situation arises the machine stops and the worker will stop the production line."

We can apply the same principles in software development in many different ways. One of the more common implementations is the use of Continuous Integration and Automated Testing.

According to the notes I took during A Day Of Bellware, if our CI server says the build is broken, we need to immediately stop working and fix the problem. I've heard this before, and will likely hear it again. However, I never really understood why people would say this. After all, as long as the problem is fixed eventually, isn't that ok? In the training that day, Scott provided a excellent visualization of why we should fix it immediately.

Waste and Rework

To steal Scott's illustrations, consider the following image to be an example of "perfect" software. All of the edges of each module (block) are well defined and it's easy stack new blocks next to existing blocks.

Figure 1. "Perfect" software

Now let's assume that somehwere in the coding process, someone accidentally causes a defect in the software. That defect could be represented by a buldge in one of the lines - as if the module was doing more than it should.

Figure 2. A defect

The individual block that has the defect may not be that bad, at first glance - or when examined on it's own. And on it's own, the defect could be fixed. Jidoka would say that you need to stop immediately and fix the problem.

So, what happens when you don't fix the problem right and and you try to stack another block to the right of the defect? Suddenly you find yourself re-shaping the next block in order to account for the problems in the previous one. Eventually, you may be able to smooth out the issues. Depending on the size of the issue, though, it may take several new modules to completely normalize things.

Figure 3. The effect of a defect

Now, 3 day, 3 weeks, 3 months, or whatever period of time later, you have a significantly larger problem to deal with. Let's go back and fix the original cause of the problem, to start with. What happens to the rest of the code that was warped around the original defect? The warped code in the rest of the system also has to be fixed. Simply removing the original bulge does not mean that the rest of the warps will magically disappear as well.

Figure 4. Removing the original bulge

How much re-work will it take to fix the rest of the system that was warped around the original problem? How much time and effort will be wasted because the original defect was not addressed immediately? These questions can only be answered in the context of your problems. If this defect is one line of code in one module, maybe it's not so bad. But if this defect is an entire module used by many other modules, the time and cost could be huge.

Conclusion: Fix It Now

If the continuous integration server says that our build is broken; or if the customer says "the software does not break, but piece XYZ isn't correct"; or ... pick a problem in your system; we need to respond as quickly as possible, in order to prevent the rest of the system from being warped by the original problem.

Under the assumption that we want to fix a defect as soon as possible - how do we ensure that we know about the problem as soon as possible? Whatever your implementation of this solution is, the solution to this problem comes down to shortening the feedback loop. If you are notified of the problem in 20 minutes vs. 20 days, there will be significantly less damage to the overall system and significantly less re-work and waste caused by the defect.

Myself and 10 other developers in my company went through a day of BDD / TDD training, with Scott Bellware, yesterday. It was a lot of fun, very challenging at times, and covered a lot of topics including an overview of Agile and Behavior Driven Development, all the way down to writing Specification Tests, doing Test Driven Development  and refactoring the model to improve readability, maintainability, flexibility, etc. I took notes via index cards (love that cool-aid) and wanted to share. I don't expect these notes to make sense to everyone. Hopefully it will spark some dialog in someone's mind and cause them to dig further.

First off - the quote of the day.

"Can I be honest with you and say that I've been wanting to touch your keyboard, all day?"

Now for my notes.

User Stories

[Role], [Goal], [Motiviation]

• As a [role], I want to [goal], so that [motivation]
• Example: As a nurse, I want to record a patients vital signs, so that I can determine their medication and care needs
• Motivation is critical - it determines how the development team understands and implements the story. It determines the user experience, how things are integrated, how the software is designed, etc.

Acceptance Criteria

• Acceptance Criteria is used to drive code, not the story, directly
• may change at any point, up to implementation
• is used to drive code design, test design, implementations, etc.
• should be spoken in domain language
• may include non-functional, technical details such as database tables, infrastructure, performance, etc
• All acceptance criteria must be met and tested / verified before a story is considered done

Specification Tests

• Test Fixture per Class is an anti-pattern (on a personal note, this problem bothered me for months before I discovered BDD)
• Context Specification or Behavior Specification testing
• When [verb] then [verb]
  • "When [verb]" is the context
  • "Then [verb]" is an observation of the behavior
• Based on Acceptance Criteria, but not "code-gen'd" from acceptance criteria

Story Estimation

• Agile Poker: uses generalized Fibonacci sequence as order of complexity
  • "?", 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, infinite
• everyone throws their estimate at same time
• if estimates have significant outliers, discussion occurs to understand why, get more detail, etc. and re-throw may happen

Entity Data vs. Aggregate Data

• Entities should never contain aggregate data
• Aggregate data is for reporting and other aggregate needs
• If you need aggregate data to process something, write an SQL query, stored proc, etc. - don't use an ORM like NHibernate
• We don't want a "Customer" entity to need 10,000 "Order" entities, to aggregate data for processing; write a query to aggregate instead
• We don't want to persist data that can be calculated / aggregated, generally (performance issues may override this)

Domain Services

• Can have dependencies on external systems
• are part of domain logic, therefore are in domain model / assembly
• are "Doers" of process that don't fit into entity and entity logic, directly
• coordination of entity logic
• can include calls to data access, logging, etc.

Continuous Integration

• Not just continuous compilation of code
• Full end to end integration of all code, components, databases, services, etc
• Full suite of integration testing including database testing
• Do not allow commits if build is currently broken
• do not allow defects to live - fix immediately, to fix build
• "Defect" is broken software, "Bug" is functional but wrong

Daily Scrum

• 3 Questions everyone answers:
  • What did I do yesterday?
  • What am I doing today?
  • What issues am I having?
• Each person should answer quickly - 1 or 2 minutes, max
• further discussion happens outside of the Scrum meeting

Productivity of Dev Team

• RAD and other non-review, non-iterative based management causes problems and loss of productivity
• we need constant review of the design to ensure good design
• shorten the feedback loop and get constant review of the design, to always improve the design, via pair programming, work cells, retrospectives, etc.
• good design will cause productivity gains in the development team beyond the capabilities of any tools

Whiteboard Diagramming vs. Details Specs

• White board diagramming and human interaction is always better than detailed documents and specs in UML
• Human interaction leads to knowledge crunching and learning, not just reading a repeating
• Take pictures, don't re-draw in UML; don't waste time with it
• Video the entire conversation is even better, so others can learn from the knowledge crunching that occurs; capture the human interaction, body language, etc.

The topic of Model-View-Presenter starter resources came up at work, today, and I like the list I pulled together. So, I thought I would share with the rest of the world. This is by no means a comprehensive list, or even "the best" list - it's only what I pulled together in 10 minutes.

The original source of MVP :
http://www.martinfowler.com/eaaDev/ModelViewPresenter.html

My own best practices for MVP, following the Passive View mentality
http://www.avocadosoftware.com/csblogs/dredge/archive/2008/02/20/787.aspx

This article should solidify a lot of what I am talking about in my best practices
http://msdn.microsoft.com/en-us/magazine/cc188690.aspx

Jeremy Miller was one of the guys to originally start pushing MVP into the ASP.NET community. He’s got a lot of good articles on his blog
http://codebetter.com/blogs/jeremy.miller/archive/2006/02/16/138382.aspx

This was the article that really solidified my understanding of the basics of MVP, back when I first started learning it
http://www.codeproject.com/KB/architecture/ModelViewPresenter.aspx

Aside from those core MVP articles, there are several key concepts that you’ll want to understand in order to really be able to present / teach MVP, including:

• Single Responsibility Principle
• Open/Closed Principle
• Dependency Inversion/Injection and Inversion of Control
• Liskov Substitution Principle

The key to these principles is really the subtlety that underlines them. For example, the Liskov Substitution Principle isn’t just a fancy name for polymorphism – it’s what makes polymorphism work. You certainly are not required to know these principles forward and backward to be able to implement MVP, but you’ll find that the question of “why?” is easier to answer to if you do know them.

Honestly, when it comes down to it, these principles are what make object oriented development work – they are how you achieve the high cohesion, low coupling, and tight encapsulation that OOD wants. There are plenty of other principles (such as Orthogonality, the law of Demeter, and others) that will help you achieve this as well; but this is a starter-list, not the all encompassing list.

One of the books I've been reading recently, that covers these topics and more, is "Agile Principles, Patterns, and Practices in C#" by Robert Martin and Martin Micah.

I highly recommend this book, not just for these topics. It's a great book, all around.