Quality must be built in, it cannot be added on. … well, ok. I’ll admit that it’s not entirely true. If you don’t mind spending exorbitant amounts of money doing rework, causing projects to be late and over budget, and possibly losing the business, then by all means, wait until after-the-fact to try and add quality onto a product. No really. It can be done. You just have to accept that the system will be rewritten. After all, isn’t that the way software development goes? Build something… try to add on to it… recognize the problems that prevent you from continuing and then justify starting the same vicious cycle over. (How arrogant are we, as software developers and development teams, to expect our customers to accept this? But, I digress…)

We already know that the real world builds quality into their products by favoring defect prevention over quality inspection. What we often don’t realize, though, is the real cost of not building in quality from the start. So, what are the costs and effects of trying to add-on quality after the fact? Before I go down that rabbit hole, though, I want you to understand how I define quality, at the moment. Without this understanding, the illustration is far less effective.

Defining Quality

I’ve run across a lot of definitions of ‘Quality’ over the years. Some of them good, some bad, some so far off from the truth that they would cause me to laugh if I didn’t know someone truly believed them. After all this time, I’ve begun to settle on my current favorite definition of Quality as: conformance to requirements.

Yes, I understand the somewhat naive focus that this definition of quality presents. However, I have yet to find as solid a foundation for quality as this. The Wikipedia entry for Zero Defect, from which I take that definition, continues to say this about quality as conformance to requirements:

“Every product or service has a requirement: a description of what the customer needs. When a particular product meets that requirement, it has achieved quality, provided that the requirement accurately describes what the enterprise and the customer actually need. This technical sense should not be confused with more common usages that indicate weight or goodness or precious materials or some absolute idealized standard. In common parlance, an inexpensive disposable pen is a lower-quality item than a gold-plated fountain pen. In the technical sense of Zero Defects, the inexpensive disposable pen is a quality product if it meets requirements: it writes, does not skip nor clog under normal use, and lasts the time specified.”

In summary - the customer is the one that defines quality. That’s a much less naive view than the definition betrays.

More Than Meets The Eye

Saying that the customer defines quality is still somewhat naive – at least on the surface. It is true that the customer is the final authority on whether or not the product is quality. However, the reality of quality in software development is that there’s more too it than just meeting the current requirements. As I’ve previously talked about, I think quality is more of a two-sided coin in software. On the one side – the primary side – you have conformance to requirements. That is, what the customer actually needs to help solve their business needs. On the other side of that coin is the technical quality, as defined by the known best practices and standards that we adhere to as individual developers, team members, companies and larger communities. (Side note: More often, I’m finding it easier to define what quality in software is not, rather than what it is.) That being said, I’ll leave the definition of quality in software engineering to those previous and future discussions.

Getting back to the original point I was making, I’d like to illustrate a situation that I’m very familiar with on the cost of not building quality in from the start.

Creating The ‘Perfect’ Failure

A long time ago, in a company far far away… [insert star wars theme]… there was a product under development. To protect the guilty innocent, we’ll call this project “ProjectX”. ProjectX’s customer was a very large entity, with lots of software products and projects that had been developed by many many software providers in the past. So, when ProjectX was first delivered to this customer and the customer said “It’s PERFECT!”, there was much rejoicing on the part of the software provider. In fact, ProjectX was deemed so perfect by the customer – having met every requirement, with less than 1% of the functionality being deemed ‘flawed, but still usable’ – that the customer declared ProjectX’s test results and acceptance as ‘the new standard’, across the board.

But there is more to ProjectX than the initial quality that the customer observed. In fact, ProjectX had a deep, dark, disturbing secret just waiting to rear it’s ugly head. That secret was the distinct lack of quality that the software provider had built into ProjectX. Everyone at the software provider knew the lack of quality – it was a necessary business decision to role ProjectX out the door as quickly as possible.

Fast forward 5 years, now. ProjectX is still a living, breathing, changing system. The customer, over the last 5 years, has requested numerous enhancements, requirements changes, and other stuff for ProjectX to do. While working on these new requirements and changes, the software provider found themselves in a sticky situation. Since they had set ‘the new standard’ with the customer – a standard so high, that they were now expected to deliver ‘on time, under budget, with perfect quality’ – the customer expected the software provider to continue meeting ‘the new standard’. Since the software provider knew that the original business decision to sacrifice quality for speed would come back to haunt them eventually, the software provider spent inordinate amounts of time trying to rectify the situation. They re-implemented parts of the system that were being modified, and introduced new architectures and standards when features needed them, and they worked 60 to 80 hour weeks for months – nay, years – on end, trying to introduce quality into ProjectX.

With each successive delivery of new features and functionality, though, the customer began to notice a distinct drop in quality. In fact, there was an alarming trend in the quality of the product, over time. The customer began to track the ratio of defective to non-defective functions, and they began to complain about the problems that ProjectX was having. Eventually, the customer’s user base refused to install the latest version of ProjectX because of the large number of known issues in ProjectX.

The customer became very upset at this new trend and many discussions with many sad users were had. And there was much sadness at the software provider. Questions were asked. Fingers were pointed. Many more discussions were had, and so-on and so-forth. The software provider, not willing to accept the new state of their software, decided to fix things… but that’s another story.

How ‘Perfect’ Failed

So how did a system that was deemed ‘perfect’ by the customer come to fail so miserably in the eyes of the same customer? The simple, honest, oh-so-hard-to-believe truth is that the software provider was unable to ‘add quality on’ after-the-fact. The software development team grew in their capabilities. They brought in new eyes and fresh expertise in development practices and architectures. They re-thought, re-structured, and re-worked so much of the system over such a long period of time. But it was never enough. There was simply too much of ProjectX that could not reasonably be changed and still meat the deadlines for new requirements, changes and enhancements.

Over time, the initial lack of quality that was built into ProjectX rotted the system, from the inside out. Because ProjectX was unable to modified to meet the new requirements without breaking the old requirements, the customer no longer saw ProjectX as a quality product. The cost of this lack of quality was tremendous – the software provider’s reputation had been tarnished, the customer did not trust the provider and did not want to use ProjectX anymore, and the software provider had a legitimate chance of failing as a company and shutting its doors.

Conclusion: Build Quality In Or Risk Losing Your Lunch Money

Creating a quality product really is a two sided coin. If you create the perfect system in the customer’s eyes, but you can’t maintain that system and add new features and functionality – where is the quality? You’ve effectively put lipstick on a pig and called it a supermodel. But if you have the ‘perfect’ engineering practices and the ‘perfect’ system that is ultra-easy to add features on to, and no one wants to buy the product because it does not solve any real business problems or provide anything that a customer actually wants or needs, then where’s the quality in that? You’ve effectively delivered a Lexus LS-400 when the customer asked for a ham sandwich.

It is imperative that quality be built in to a product from the start. We know how to do this. We are (or should be) standing on the shoulders of giants. We only need to keep in mind both sides of the coin when creating the needed quality. Customer quality first, with engineering equally as important after the customer says ‘PERFECT!’.

This is not just a scary bed time story and this is not just a bunch of ‘ivory tower’ fluff. “ProjectX”, despite a fictitious name, was a real project at a real company and the story I told is the boiled down version of really happened. In fact, it has happened at least once in every job I’ve had as a professional software developer. I know the cost of not building in quality from the start. I’ve seen my friends, coworkers and bosses lose their lunch money. It’s not a happy scene.

I read Sergio’s post on “Javascript, time to grok closures” today and it was very enlightening. Overall, it helped me to understand closures more than I previously had – not just in Javascript, though. I put together a quick sample on closures in C# to illustrate the same behavior that Sergio is talking about in the ‘Closures can be tricky’ section of his post. I’m happy to see C# is behaving the same way as Javascript, in the case of ‘value’ types in closures. This probably isn’t new to anyone that understands closures already. It’s new to me, though, and seems to be a fairly important concept to understand when using anonymous methods (lambda expression or otherwise) and closures.

Illustrating The Closure Scope For Value Types

Here is the sample code I put together, in unit test form, to illustrate the same behavior that Sergio is pointing out.

[TestFixture]

public class When_referencing_a_value_type_variable_from_the_parent_scope_directly_in_a_closure : ContextSpecification

{

   private readonly IList<Func<int>> theActions = new List<Func<int>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           Func<int> foo = (() => { return i; });

           theActions.Add(foo);

       }

   }

   [Test]

   public void Should_reference_the_original_value_in_the_parent_scope()

   {

       //since the value of "i" in the original scope is now 4 (having gone through all of the loop's iterations)

       //every value in the execution of the "foo" anonymous function will be 4

       int expectedValue = 4;

       int value0 = theActions[0]();

       Assert.AreEqual(expectedValue, value0);

       int value1 = theActions[1]();

       Assert.AreEqual(expectedValue, value1);

       int value2 = theActions[2]();

       Assert.AreEqual(expectedValue, value2);

       int value3 = theActions[3]();

       Assert.AreEqual(expectedValue, value3);

   }

}

[TestFixture]

public class When_passing_a_value_type_variable_from_the_parent_scope_to_a_method_called_by_a_closure : ContextSpecification

{

   private readonly IList<Func<int>> theActions = new List<Func<int>>();

   private void dooFoo(int value)

   {

       Func<int> foo = (() => { return value; });

       theActions.Add(foo);

   }

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           dooFoo(i);

       }

   }

   [Test]

   public void Should_create_a_copy_of_the_variable_and_retain_the_original_value()

   {

       //since we called a method, passing "i" from the original context into that method,

       //the copied value was sent to the "foo" anonymous function, meaning that we got all

       //of the numbers - 0, 1, 2, and 3 - in the anonymous "foo" functions.

       int expectedValue = 0;

       int value0 = theActions[0]();

       Assert.AreEqual(expectedValue, value0);

       expectedValue = 1;

       int value1 = theActions[1]();

       Assert.AreEqual(expectedValue, value1);

       expectedValue = 2;

       int value2 = theActions[2]();

       Assert.AreEqual(expectedValue, value2);

       expectedValue = 3;

       int value3 = theActions[3]();

       Assert.AreEqual(expectedValue, value3);

   }

}

Illustrating The Closure Scope For Reference Types

It gets a little more interesting when dealing with reference types. The same basic statements can be made for reference types, but we have to remember that we’re now dealing with what are essentially pointers, not values. So, when a reference type variable gets copied, it still points to the same object on the heap. Thus, when dealing with reference types in closures, the scope of the variable that creates the reference type suddenly becomes much more important.

To illustrate the important of variable scope when dealing with closures, I’ve created some examples that scope a small reference type (class) in a few different ways. I’ll use this class definition as the reference type in all three examples.

public class SomeClass

{

}

Closure Of A Reference In Parent Scope

[TestFixture]

public class When_creating_a_closure_for_a_reference_that_is_part_of_the_parent_scope : ContextSpecification

{

   SomeClass someClass;

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           someClass = new SomeClass();

           theValues.Add(someClass);

           Func<SomeClass> foo = (() => { return someClass; });

           theClosureValues.Add(foo);

       }

   }

   [Test]

   public void Then_all_closures_should_reference_the_last_instance_of_the_variable_from_the_parent_scope()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreNotSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreNotSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreNotSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this example, we’ve declared the “someClass” variable outside of the for-loop. This creates a variable that is scoped outside of the loop, and in this case outside of the Context() method. As long as this variable is scoped outside of the for-loop, we will only have a single variable to create our closure on. We see that the foo() anonymous method encloses someClass, causing it to stay in scope. What we don’t see, though, is that the foo() anonymous method will not be evaluated until the observations (just before the asserts) are executed. This means that every foo() method will have a reference to the same instance of “SomeClass”, not the individual instance that was created inside of the for-loop.

Closure Of A Reference That Is Scoped Within The Loop

[TestFixture]

public class When_creating_a_closure_for_a_reference_that_is_scoped_within_the_forloop: ContextSpecification

{

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           SomeClass someClass = new SomeClass();

           theValues.Add(someClass);

           Func<SomeClass> foo = (() => { return someClass; });

           theClosureValues.Add(foo);

       }

   }

   [Test]

   public void Then_all_closures_should_have_their_own_reference_from_the_loop_iteration()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this example, we’ve declared the “someClass” variable inside of the for-loop, and is thus, in the local scope of the for-loop. Since the foo() anonymous method is creating a closure on a variable that is scoped to the for-loop, we can be assured that each foo() method will point to the “SomeClass” instance that was created in the for-loop. Our closure’s scope is now limited to the for-loop.

Closure Of A Reference Pointer Copy

[TestFixture]

public class When_creating_a_closure_for_a_copy_of_a_reference_pointer : ContextSpecification

{

   SomeClass someClass;

   private readonly IList<SomeClass> theValues = new List<SomeClass>();

   private readonly IList<Func<SomeClass>> theClosureValues = new List<Func<SomeClass>>();

   private void dooFoo(SomeClass value)

   {

       Func<SomeClass> foo = (() => { return value; });

       theClosureValues.Add(foo);

   }

   protected override void Context()

   {

       for (int i = 0; i <= 3; i++)

       {

           someClass = new SomeClass();

           theValues.Add(someClass);

           dooFoo(someClass);

       }

   }

   [Test]

   public void Then_all_closures_should_have_their_own_reference_to_the_copied_reference_pointer()

   {

       SomeClass expectedValue0 = theValues[0];

       SomeClass value0 = theClosureValues[0]();

       Assert.AreSame(expectedValue0, value0);

       SomeClass expectedValue1 = theValues[1];

       SomeClass value1 = theClosureValues[1]();

       Assert.AreSame(expectedValue1, value1);

       SomeClass expectedValue2 = theValues[2];

       SomeClass value2 = theClosureValues[2]();

       Assert.AreSame(expectedValue2, value2);

       SomeClass expectedValue3 = theValues[3];

       SomeClass value3 = theClosureValues[3]();

       Assert.AreSame(expectedValue3, value3);

   }

}

In this last example, we can see that the “someClass” variable is again scoped outside of the Context() method. However, notice that we are now calling a “dooFoo()” method inside of the loop, instead of creating the closure locally. The effectively means that we are not creating a closure of “someClass” anymore. When someClass is passed into the dooFoo method, a copy of the reference is made – that is, we have a new variable with a new scope; this new variable just happens to be pointing to the same object as someClass, on the heap. Since we are now creating a closure around the “value” parameter of dooFoo, the scope of the closure has changed and acts the same as the closure of a reference in that is scoped with the loop. Each foo() anonymous method, with it’s closure, will now have it’s own instance of “value” to reference – the reference that was created in the scope of the for-loop, and then copied into the “value” parameter.

In “Closing” (pun intended )

It’s important to keep the scope of the variables in our closures in mind. If we scope our closures incorrectly, we can have some strange bugs in our system. Understanding the scope of your closures does require us to know a little bit of computer science, though – stack vs. heap, reference vs. value type – but that’s not a bad thing.

(If you see something wrong in my explanations or think my spec/observation names should be changed, let me know. I tried to make it clear, but am not sure that I did a good job.)

I heard the phrase “one team, one aim” while listening to the audio book version of “Extreme Toyota”. This is a phrase that marks part of the core philosophies of Toyota, according to the book. There are many different philosophies that underlie this specific phrase and I’m sure there is a very large amount of text that can be written about it. Rather than jumping off of that deep end without a greater understanding than I currently have, though, I wanted to point out one very specific word in this phrase: “aim”.

Wiktionary has a definition of aim that appears to fit very well in this phrase: “Intention; purpose; design; scheme.”

So, why use the word “aim”, here? Why not talk about goals? Why not deal with something more concrete? I think the answer to this is found in one of the core principles of Toyota: Continuous Improvement. A goal is a concrete, fixed point to be reached. Although a goal can change, once it does change, it once again becomes a fixed point to be reached. If Toyota talked about goals then they would inherently be limiting themselves to such fixed points of achievement. By using the word “aim” instead of “goal”, they are illustrating the true intentions of Continuous Improvement. That is, they are not setting their sights on attainable goals. Rather, they are taking a philosophical, yet very practical and daily routine of continuously improving in all aspects.

This does not mean Toyota works without goals. Quite the opposite, really. They create some of the most ambitious goals that anyone can imagine - “impossible goals” as talked about in the Extreme Toyota book. However, these goals are always aligned with the company’s vision, their aim. Toyota recognizes that a single goal is never a place to stop, having accomplished something, but is only a starting point allowing them to define the next goal and begin that leg of the journey.

As I’ve talked about before, we should be focused on the journey – the aim of where we are heading - constantly adjusting that aim as we obtain new information. We then set goals to help us move in that direction, knowing that we are truly on a journey, not looking for a place to stop after a single, fixed point of achievement.

I have a service object with an interface explicitly defined for it. I like this because it let’s me unit test the things that need the service without having to worry about the implementation detail of the actual service.

public interface IMyService

{

  public void DoSomething();

  public void AnotherThingHere();

}

When I get to the implementation of this service and I start specifying the behavior through my specification/tests, I create a class that has a dependency on another interface – ISomeRepository. This repository is used in both methods of the actual service implementation.

For the “AnotherThingHere” method, I end up with several specification/tests because that method has some good business logic in it.

For the “DoSomething” method, though, the real implementation is only a pass-through to the repository and my specification/test ends up looking like this:

[TestFixture]

public class When_doing_something: ContextSpecification

{

 ISomeRepository repo;

 public override void Context()

 {

   repo = MockRepository.GenerateMock<ISomeRepository>();

   IMyService myService = new MyService(repo);

   myService.DoSomething();

 }

 [Test]

 public void Should_do_something()

 {

   repo.AssertWasCalled(r => r.DoSomething());

 }

}

I know this specification is necessary because I am using the “DoSomething” method of IMyService in other parts of the system. I think there is value in having an IMyService interface explicitly because it simplified the specification/tests for the parts of the system that need to use it, and decoupled the system to a point that made it much easier to code and change.

So my question is, do you see any real value in a specification/test name like “When doing something, should do something”? or should I be looking at this test from a different “style” or perspective?

I think this specification/test is valuable, but I also think the test name and observation name are silly since they say the same thing. Advice? Different naming suggestions? What am I missing or just not seeing? or is this ok and I’m just running on 25% brain power due to lack of sleep today?

Update: There’s a wealth of knowledge out there that I just haven’t been aware of until now. Thanks to the many commenters of this post, other posts, and other conversations for the links and info. Here is a list of items that I’m finding to be very helpful in understanding what is only a surface level “validation” problem:

• Validation in a DDD World
• A response to Validation in a DDD world
• DDD: Entity Validation and Command/Query Separation
• DDD: CQS and Bounded Contexts
• Greg Young Discusses State Transitions in Domain-Driven Design and DDD Best Practices (InfoQ Video interview)
• DDD: Command Query Separation as an Architectural Concept

Update 3/3/2009 – additional info in the continuing conversation.

• Who Knew Domain Validation Was So Hard?
• Validation and Domain Driven Design

Update 3/11/2009

• DDD: Validity, Consistency and Immutability
• DDD: Invariants or Contextual Validation?

There’s a whole host of stuff out there linked from these articles and posts, as well. It’s obvious now that I have a lot to learn and think about, and I think I have a better sense of where I need to start looking. I also think that the ideas Sean Biefeld expressed about never letting an entity be in an invalid state are more reachable than I previously thought.

----------------------------------

This post started as a comment in Sean’s post on Entity Validation Ideation but quickly grew. I’m hesitant to post separately because I believe the conversation should continue along the lines of what he was originally saying. There’s just so much junk tossing around in my head, though, I didn’t think it would be appropriate to post a 50 page diatribe in a comment.

A while back, I asked the question “Where does required info validation belong for an entity?” There were a lot of good answers to that question and my team and I took some very interesting directions based on those answers. However, I still haven’t had the ‘validation’ question completely answered in my mind. Fortunately, I’ve got the brains of my fellow Los Techies (and coworker in this case), to keep me from giving in to the entropy that wants to keep me mediocre.

I love the theory that Sean is talking about - especially when stated as 'proactive' vs 'reactive'. I've had a hard time with this in practice, though. In the last few weeks – days, really - it seems to me that we need to have both proactive and reactive validation in our applications.

Being Proactive

The domain model that I create wants to prevent an object from being invalid, so I require a minimum set of information in my constructors, etc etc. This helps me prevent an object from ever being created in an invalid state. this is the proactive approach. I love this approach and I use it a lot.

Oh Wait, I Need Reactive

The proactive recently failed me, though. I had ‘the perfect’ domain model with the all the validation and rules composed in my entities, preventing the information from being added to the model if it was not valid. However, when I got to the user interface, I realized that I had a major design problem. The user interface is not 'all or nothing' scenario. The user is allowed (expected, really) to only provide one field of information at a time. After all, you can't type into more than one text box at a time - unless you have multiple keyboards and multi-input capable systems. So, when a user is adding information to the inputs of the form, we need to have a reactive approach. We need to provide real time validation of what the user is putting into the system, so that they can be immediately notified of any errors in what they are providing. This reactive approach is needed because the user interface is entirely reactive to the user's actions.

Proactive AND Reactive?

At the same time that we need both a proactive and a reactive solution, we want to adhere to the "Don't Repeat Yourself" principle, encapsulating the validation logic into the appropriate location(s). If we require reactive validation in the user interface, and proactive validation in the domain model, how do we avoid breaking DRY?

On the surface, proactive and reactive validation seem to be at odds with each other – at least, they do in my mind. I don't think they need to be, or should be, though. We just need to find the higher level order in which we can enable both. Unfortunately, I haven't been able to do this yet – at least not in a way that really seems to satisfy me. I'm curious to know how others approach this situation. How do we balance this triad of need: Proactive, Reactive, and DRY?

I need some fresh perspectives on this, and I think Sean is starting out with some great ideas. Now if I can just learn to throw away my own bias to how I’ve already been handling this, maybe I’ll be able to learn something.

I found the Motivator this morning. It lets you create your own motivational pictures. So, here’s my first run at creating the SOLID software development principles in motivational picture form. I ‘borrowed’ the images from google image search results. I hope you find them to be as fun as I do! I have them all hanging up in my team room, already.

(Update: I never expected the response to this post to be so great! There have been many inquiries about prints, high res versions, etc. As I said in the comments below, the source pictures that are linked via the thumbnails are all I have. There does appear to be some effort to produce high res versions, though. See the comments for more info!)

Update: Due to the continuous request for prints, posters, calendars, etc, we (LosTechies) are looking into what it would take to get these turned into high quality prints of various types. I don’t have any detail yet, but I am hoping to have some good info on this, fairly soon.

SOLID

Software development is not a Jenga game.

(This one was created by Mark Nijhof’s. He posted it via twitter and I’m borrowing it for my own collection.)

Single Responsibility Principle

Just because you can, doesn’t mean you should.

(Update: I knew I had seen this Swiss Army knife in a Single Responsibility post before. Gabriel reminded me where.)

Open Closed Principle

Open chest surgery is not needed when putting on a coat.

Liskov Substitution Principle

If it looks like a duck, quacks like a duck, but needs batteries – you probably have the wrong abstraction

Interface Segregation Principle

You want me to plug this in, where?

Dependency Inversion Principle

Would you solder a lamp directly to the electrical wiring in a wall?

I’ve written a lot of specification tests like this in the last three years, from a UI / Workflow perspective, with Model-View-Presenter as my core UI architecture:

[TestFixture]

public class When_starting_some_process()

{

 IMyView view;

 MyPresenter presenter;

 [Setup]

 public void Setup()

 {

   //...setup code and execute stuff for the test here

   view = MockRepository.GenerateMock<IMyView>();

   presenter = new MyPresenter(view);

   presenter.StartSomeProcess();

 } 

 [Test]

 public void Should_attach_the_view_to_the_presenter()

 {

   presenter.View.ShouldNotBeNull();

 }

 [Test]

 public void Should_show_something()

 {

   view.AssertWasCalled(v => v.ShowSomething(something));

 }

}

The Devil In These Details

I had one of those ‘aha!’ moments yesterday where several of my nagging suspicions and annoyances at writing specification tests like this example finally gelled into a coherent understanding. That understanding is easily stated by saying that the test, “Should_attach_the_view_to_the_presenter” is invalid and should never be written. There are a number of reasons for this.

1. In practicing BDD, the technical jargon that is leaking into the test is irrelevant to the real value that is intended with this specification
2. In practicing any form of TDD, the implementation detail of attaching a view to a presenter creates a brittle test – I care about the API and how the system really works, not a very technical, implementation detail like this.
3. Exposing the “View” property of the Presenter object is a violation of encapsulation and an ‘over-intimate’ smell. My test has far too much fine detail, granular knowledge of what’s going on behind the scenes, to really be of any practical value
4. Finally – and possibly outweighing all other reasons combined – it’s simply not necessary.

Look at the second test: “Should_show_something”. It’s actually using the view. Presumably, the “StartSomeProcess” method on the presenter is going to call a method on the view – that’s why we are asserting that the method on the view was called. If this is true, then it can be safely assumed that not having a view attached to the presenter would throw a null reference exception when trying to call that method on the view.

If not having the view attached to the presenter results in a null reference exception for the other test, then we have a valid reason for that test to fail. We certainly can’t say that the view’s method was called when the view is null. Therefore, testing to ensure that we have a view attached to the presenter is a duplication of effort. We’re only proving what we have already proved transitively, via another test.

Beauty And Meaning In Simplicity

Assuming that this is all true, we can remove that test entirely. Our specification now looks like this:

[TestFixture]

public class When_starting_some_process()

{

 IMyView view;

 MyPresenter presenter;

 [Setup]

 public void Setup()

 {

   //...setup code and execute stuff for the test here

   view = MockRepository.GenerateMock<IMyView>();

   presenter = new MyPresenter(view);

   presenter.StartSomeProcess();

 }

 [Test]

 public void Should_show_something()

 {

   view.AssertWasCalled(v => v.ShowSomething(something));

 }

}

It’s one less test to deal with, yet is as expressive and meaningful. In fact, I would say it is more meaningful because we have avoided all of the problems I listed. I’m reminded of a quote by Alan Cooper:

“No matter how beautiful, no matter how cool your interface, it would be better if there were less of it.”

Although I think he was specifically addressing User interfaces in this quote, it is applicable to all interfaces – programmatic, user, etc. Simplicity and subtleness is the key. I know I’m not the first person to talk about this problem, why it’s a problem, or the solution. I’m only claiming that I finally had that ‘aha!’ moment and realized why so many others have talked about this before.

Wow! I’m actually writing code!

I thought this little snippet would be useful for anyone that is binding a custom collection of objects to a Telerik MultiColumnComboBox. If you want to get the selected item as your actual class – the custom class that you wrote and bound as a collection to the list, you can use this basic code:

private void MyComboBox_SelectedIndexChanged(object sender, EventArgs e)

{

   GridViewDataRowInfo rowInfo = MyComboBox.SelectedItem as GridViewDataRowInfo;

   if (rowInfo != null)

   {

       MyClass myClass = rowInfo.DataBoundItem as MyClass;

       //do something with the 'myClass' variable, here.

   }

}

In the manufacturing world, you would never find a company that assembles a bunch of parts into a final product before inspecting any of the individual parts, and they would not wait until the end of the assembly line to test for the quality of the product. The very notion of waiting until the product is “done”, to test it, would be appalling. How much time and money would be lost trying to figure out why something didn’t fit together properly, why it didn’t work, and why there was a quality issue with the final product? Rather, we see the manufacturing world taking an active role in preventing defects. Yes, they still do a final quality inspection, but the primary means of ensuring a quality product is delivered is not by waiting until the product is assembled to test it. They build quality in from the start and maintain that quality throughout the manufacturing process.

Prying Open The Case

Imagine the inner workings of the phone to the left. This is a very complex piece of technology. Do you think Cisco would wait until they have assembled this phone and then try to pry open the case so that they can insert a set of electrodes to test and see that the circuit board is connected correctly? I certainly hope they don’t. Instead, when a manufacturing company is building something – anything – they start with the idea of preventing defects, not waiting until they are identified and correcting them. Many companies have an active Zero Defects policy where defect prevention is paramount and quality inspection is almost just a verification of what they already know – that the product is defect free.

When a part is stamped out, formed, molded, or otherwise created, it is done so to an exacting specification. After the part has been created, the part is then tested against the same specification to which it was originally built. If the part does not fall within the tolerance and guidance of the specification, it is scrapped and a new one is made. If a series of parts are found to be out of specification, it’s usually a sign that something in the process, tooling, or other portion of the manufacturing process is not right. When this happens, they fix the cause of the problem – whether the machines need to be calibrated, the people running the machines need better instructions or whatever the cause is. In the end, the specifications for the parts were used to create the part, identify whether or not the part was up to standards, and decide whether or not to keep that part.

What’s more, the manufacturing company doesn’t wait until after they start creating parts to create the specification. Rather, they take the time to properly engineer the specifications up front. They take measurements, create prototypes with varying specifications to see what works best, record the success and failure rates of the various specifications that are tried, and use other design and engineering principles to scientifically calculate the exacting specifications that will be used to produce the parts. This occurs at all levels of the product’s design and creation. For every resistor, capacitor and microchip that builds a circuit board, each one of them has their own specifications that have been carefully engineered. If any single capacitor does not meet the specifications, it is not sent to Cisco with the hopes that it works anyways. Only when all of the specifications of each part are met will they solder the parts to the circuit board, creating a subassembly.

When a subassembly is created, it also has a specification to which it was built. The subassembly then undergoes the same quality assurance process – verification that it meets the specifications and operational requirements. The process continues from here – each subassembly gets connected to a larger system which is built to a set of specifications, with rigorous testing of the larger system as it is built, ensuring it meets the specifications. When the final phone is assembled, we don’t have to worry about whether or not a specific capacitor is soldered to the correct location – we don’t have pry open the case on this phone and insert a set of electrodes to see if the electrical current is flowing correctly. Instead, we only need to plug this phone into the correct connections (an Cisco IP phone system in this case) and verify that the phone actually performs all of it’s functions, according the functional specifications of the phone. There simply is no need to verify the capacitor that was used in the very first step. We know it works because it was built in a system that actively prevents defects.

The manufacturing world is obsessed with testing. They are willing to test from the lowest possible levels of the system, out to the end-product and the behavior that is expected. They do this because the consumers of manufactured products demand perfect. Why, then, are so many software development companies so willing to only test from one end of the process? To only test once, and only from the user interface, just before the product is shipped?

A Specification By Any Other Name

Unfortunately, the software development industry as a whole, is years behind the manufacturing industry. Our definition of quality and success are often skewed and we may consider fifty or more known bugs in a system of moderate to large size to be acceptable. It doesn’t have to be this way, though. We have the technical capabilities of following in the footsteps of the manufacturing industry, and we should.

I’m sure there would be no small number of people that would say we already employ the use of specifications in software development. After all, that’s what the requirements gathering phase is for, right? So many software development companies have put so much effort, time and money into the process of producing a piece of paper that can be understood by humans, and labeled this a specification. The problem we face with paper, though, is how to effectively verify the software against what the paper says. How do we verify that series of software lines and I/O statements that are understood by a computer have actually implemented the human readable text on the paper?

We are fortunate, actually. We don’t have to accept a Word document or a piece of paper as the specification to build to. We have the ability to create executable specifications! We can, and should, be creating specifications that can measure and verify our code. Most people call it test driven development (TDD). Some call it Behavior Driven Development (BDD). I like to think of it as Specification Driven Development (SDD? Not sure if that really exists. And really, do we need another xDD acronym?). We write code that exercises our code in the form of unit tests, integration tests, functional tests, acceptance tests, or whatever you want to call them.

We’re Not Just Stamping Out Parts

One of the major problems that I have with the manufacturing/software development analogy is the obvious statement that we don’t stamp out the same parts over and over again. In spite of my previous comments on this analogy, I now think that we are more analogous to a specific part of manufacturing than I had previously understood. A more accurate representation of software development in the manufacturing world is new product design and development. The parallels work quite well from this perspective. I am not going to expound on this in great detail at this point. It should suffice to say, for the moment, that the process of product development described in Wikipedia is a fairly accurate representation of what we go through for the average software development project.

When a manufacturing company is working on a new product, they once again don’t stamp out parts without knowing what they are doing. Many different parts may need to be tested, many different designs may need to be tried, but every one of these is still built to a specification. The major difference is that the specifications used are expected to change over time, until the final specification for the final pieces are accurate enough to produce a production-ready prototype.  The same notions can be applied to the software development processes in TDD, with some additional benefits.

Built To Specs, Regression Tests And Change

Change happens. It’s a simple fact of software development. A customer thought they wanted X, but in reality they needed X-1/B – not quite what we originally thought. When this happens, we once again have a significant benefit created by our executable specifications. We only need to identify those specifications that are now wrong, correct them, and change the affected portions of the system to match the new specifications.

Our ability to change is direct evidence to one of the many benefits of TDD: regression tests. Every specification that we write becomes a regression test the moment we fulfill that specification’s requirements. With this in mind, we can work with an almost reckless abandon, free to add features, remove features, fix bugs (because let’s face it – we’re still going to find some issues somewhere in the system) and refactor the system to a higher standard, all without worry of breaking the system. We can act with this level of confidence because we have a safety net in our regression tests. If (when) we do break something in our new efforts, we will be notified the moment we re-execute our specifications – that is, run our regression tests. A specification test will fail and we will have a clear indication of what failed and why. This deep insight into the system gives us even further confidence in correcting any issues that we introduce. When a failed specification test tells you exactly which value from which class is wrong, and the context in which that class was executed is known (the exact input and expected output), pinpointing the problem becomes a rote process. Fixing the issue becomes relatively simple, and we begin to see true productivity improvements in our processes.

Start With Quality, End With Quality

Our industry is currently suffering from a lack of quality. We ship horrendously bad user experiences in products that are late and well over budget, yet we call this a ‘success’. It doesn’t have to be this way. If we change our perspective and start to take some cues from the manufacturing and product design and development world, we can dramatically increase our effectiveness as software developers. We can create high quality, low cost solutions like the world expects from manufacturers. Built to specification is certainly not a silver bullet. It is, however, the definition of quality in a Zero Defect environment.

By employing a built to specification mindset in our software development efforts, we can start with quality and maintain that quality throughout the life of our projects. This is the same process that is undertaken when a manufacturing company is working on new product design and development. It works well, it’s a proven process, and most of all – it makes sense. Build your software to specifications. Just make sure they are executable specifications.

I recently saw this question asked and answered in an after-project retrospective paper.

Was The Project A Technical Success?

While I am not trying to address who asked the question or what the answer was, the question on it’s on own, is flawed in my mind. Or, at least when taken out of context, it has become somewhat of a fallacy by standing on its own.

Not ‘Technical’, But ‘Operational’

The question we should be asking first is not about ‘technical’ success, but ‘operational’ or ‘business solutions’ success:

Was The Project An Operational Success?

That is to ask, did we effectively reduce or solve the stated business problems to an acceptable level, allowing the users of the application – the operational people - to be more accurate, efficient, productive, responsive, … etc, in their job? Only when we can answer “yes” to the project being an operational success, and provide sufficient evidence of this success, should we ask if the project was a technical success.

If we created the world’s most perfect architecture, design and implementation, but the project was a DOA failure because it did not solve or sufficiently reduce any stated business problems, resulting in no one to buy it or use it; what good would it be to have the perfect system from a technical standpoint?

The counter-perspective, though, is just as important. If we have managed to solve or sufficiently reduce the stated problems then we need to ask if we did so in a manner that allows us to maintain this system for the long-term life of it. Were we successful in our technical implementation of the business solution, thereby allowing us to continue solving additional / ancillary business problems, easily and efficiently?

Two Sides Of The Same Coin

We can’t sacrifice the needs of the business to create what we believe is a successful technical implementation. Yet, we can’t sacrifice the technical implementation just to satisfy the needs of the business. There is a marriage between the two – they are (or should be) two sides of the same coin. If we fail on either aspect and are not able to recover quickly, we will eventually fail as a project / product team.