Barbarian Meets Codingbarbarianmeetscoding

WebDev, UX & a Pinch of Fantasy

Write 30% Less Test Code With AutoFixture

I have been following Mark Seemann for a while ‘cause this guy is as smart as it gets. He wrote a book on dependency injection, released two awesome courses on unit testing in Pluralsight, and has written the AutoFixture unit testing library (with many other contributors).

AutoFixture is a .NET library that helps you write unit tests with less code by providing:

  • Test data generators for primitive and complex types (in a constrained non-deterministic way - the data generation algorithm can be customized)
  • Test data builders for any type
  • AutoMocking of dependencies
  • I repeat, A U T O M O C K I N G of dependencies

and in words of Mark Seemann:

AutoFixture is an open source framework for .NET designed to minimize the ‘Arrange’ phase of your unit tests. Its primary goal is to allow developers to focus on what is being tested rather than how to setup the test scenario, by making it easier to create object graphs containing test data.

What does this means in terms of that application you are developing right now if you were to adopt AutoFixture just this very moment?

  • Write less unit test code

    • Bye writing mother objects
    • Bye writing your own test data builders
    • Bye to manually providing dependencies for your system under test
  • But wait Joe! There’s more!

    • By using a SUT (system under test) factory we effectively isolate all tests from changes in the SUT constructor. You will probably have handled this via factory methods, builders or fixture objects. But with AutoFixture and automocking, adding a new dependency to a constructor will not break any test!
    • Being able to automock dependencies means that you don’t need to explicitly mock/stub dependencies that are not used in a given test, which in turns means that tests become more readable, intentional and deliberate. The code that it is there takes active part in the test. All crud is removed, gone, caput.

Installing AutoFixture

You can easily start using AutoFixture in your project via Nuaet:

>Install-Package AutoFixture
>Install-Package AutoFixture.NUnit2
>Install-Package AutoFixture.AutoRhinoMock

Note: I am using Nunit and RhinoMocks in this article but there is support for many other testing frameworks and mocking libraries.

Where each of these packages consist in the following:

  • AutoFixture: this is AutoFixture itself
  • AutoFixture.NUnit2: this is an improved integration between AutoFixture and NUnit that allows AutoFixture to inject test data and SUT directly in NUnit Test methods
  • AutoFixture.AutoRhinoMock: this allows auto mocking of dependencies via Rhino.Mocks

Using AutoFixture

I am going to go fast like lightning through some of the interesting features provided by AutoFixture, brace yourself:

Auto Mocking SUT Dependencies

public class Calculator{
	private readonly ILogger logger;
    private readonly IEmailSender emailSender;
    private readonly IArgumentParser argumentParser;
    private readonly IBellsAndWistles bellsAndWistles;

    public Calculator(ILogger logger, IArgumentParser argumentParser, IEmailSender emailSender, IBellsAndWistles bellsAndWistles){...}
    public int Add(string arguments){
	   Tuple<int,int> operands = argumentParser.Parse(arguments)
       var sum = operands.Item1 + operands.Item2;

       logger.Info(string.Format("I just added a couple of numbers {0} + {1} = {2}", operands.Item1, operands.Item2, sum));
       emailSender.Send(new AdditionWasDoneNotificationEmail(operands, sum));
       bellsAndWistles.Alarm(operands, sum);

       return sum;
	}
}

[TestFixture]
public class CalculatorTests{

    private IFixture fixture;

    [SetUp]
    public void SetUp(){
       fixture = new Fixture().Customize(new AutoRhinoMockCustomization());
    }

    [TestCase("1 1",1,1,2)]
    [TestCase("2 2",2,2,4)]
    [TestCase("3 2",3,2,5)]
    public void AddTwoIntegers_ShouldPerformAddition(string arguments, int op1, int op2, int expectedSum)
    {
       // Arrange
       var argumentParser = fixture.Freeze<IArgumentParser>();
       argumentParser.Stub(p => p.Parse(arguments)).Return(new Tuple<int,int>(op1, op2));

       var calculator = fixture.Create<Calculator>();
       // Act
       var sum = calculator.Add(arguments);
       // Assert
       Assert.That(sum, Is.EqualTo(expectedSum));
    }

    // more tests

}

Test Builders For Any Type

var camilla = fixture.Build<User>()
					  .With(u => u.FirstName, "Camilla")
                      .With(u => u.SecondName, "For Great Justice")
  				      .Without(u => u.Roles)
					  .Without(u => u.ParentCompany)
                      Create(); // other fields filled with data generated

Generating Test Data

// for complex types
var camilla = fixture.Create<User>(); // all fields are filled with data generated
// for primitive types
var number = fixture.Create<int>();

Note For The Intrepid: AutoFixture doesn’t work well if there are cycles in the object graph. If a complex type has cycles, you will need to tell AutoFixture to leave that property alone (OmitAutoProperties) or set it explicitely.

Integration with NUnit

If we do a full integration with NUnit we can remove the Arrange stage and inject the test data directly as arguments:

[Test, AutoData]
public void IntroductoryTest(int expectedNumber, MyClass sut)
{
    // Act
    int result = sut.Echo(expectedNumber);
    // Assert
    Assert.Equal(expectedNumber, result);
}

And even auto mock SUT dependencies:

[Test, AutoData]
[AutoRhinoMockData]
public void IntroductoryTest(int expectedNumber, MyClass sut)
{
    // Act
    int result = sut.Echo(expectedNumber);
    // Assert
    Assert.Equal(expectedNumber, result);
}

// in this case you need to define a new attribute
internal class AutoRhinoMockDataAttribute : AutoDataAttribute
{
    internal AutoRhinoMockDataAttribute() : base(new Fixture().Customize(new AutoRhinoMockCustomization())){}
}

Mind Officially Blown Away. Booom… Boooom… Mindbloooown with AutoFixture Booom… Boooom… Booooom…

It even has testimonials. Open-source library with testimonials?!? Whatz?!?:

“I’ve introduced AutoFixture to my developers (at www.gab.de ) some time ago. We’ve been using it successfully with xunit in multiple projects all across the .NET technology stack. We also use it for feeding dummy data to the UI when developing prototypes. That saved us quite some time. Florian Hötzinger, GAB Enterprise IT Solutions GmbH

“I have used AutoFixture for 3 years, it’s a vital tool in my TDD toolbox, a real time-saver. Setting up maintainable and robust unit tests with AutoFixture is easy and straightforward - highly recommendable” Mads Tjørnelund Toustrup, Senior .Net Developer

“Autofixture is more than just another test data generator. It helps me to write tests faster, which are robust against changes in my production code. Moreover, with Autofixture I can focus the tests on the behavior I want to check which why they are easier to read and understand.” Hendrik Lösch, Saxonia Systems AG

It is time to Experiment Thyselve! Go NuGet Install-Package AutoFixture Now!

3 Things I Learned From Seductive Interaction Design, Creating Playful, Fun and Effective User Experiences - A Barbaric Book Review

With Barbaric Book Reviews I bring you interesting reviews and useful insights from awesome books that I have read. I also bring myself the magic of everlasting memory so that I don’t forget these tidbits of knowledge as I grow old and wither.

I should read more UX books. We! [Developers] should read more UX books. They are filled with wondrous jewels of information about the arcane arts of psychology and human behavior, of inestimable value when crafting effective and engaging user experiences. They are also beautiful, I haven’t seen a UX/Design book that wasn’t in itself a wonderful and inviting artpiece of a book. (ehrr… Jaime, always sooo enthusiastic)

We recently started a book club and a library at work. In the book club we read a book a month and afterwards we discuss it (cool), in the library we have shared our books with each other for the betterment of the Universe and for great justice. That’s how I got to borrow this great book titled Seductive Interaction Design: Creating Playful, Fun and Effective User Experiences by @stephenanderson from my good friend (and UX whiz) Erik Claesson and Voilà! The article!

Read on →

Dev Talk Monday: Architecting Large JavaScript Single Page Apps With Knockout.js

Dev Talk Monday is the series that brings you awesome dev talks every Monday

This weekend I had the chance to see a superb talk by Steven Sanderson on building large scale SPAs (Single Page Application) with Knockout.js. This talk is invaluable if you are spending any amount of time working with Knockout.js and are looking for tips on how to bring your front-end to 2014. You will learn:

Pretty awesome.

Also pretty awesome the fact that you can see all ndc 2014 talks on vimeo (and all ndc talks from all time back to the prehistoric age)

Thoughts on Unit Testing And TDD: Test Behavior, Not Implementation

Ladies and gentlemen: The blog post you are about to read is based on a true story/e-mail. Only the names of classes and methods have been changed to protect the privacy and innocence of the source code. The facts remain unchanged and the contents unadultered…

Disclaimer: I love automated testing and I have been a practicioner of TDD since 2010.

When I refer to unit tests in this article I use the most strict definition of unit tests, that is, test a unit of code - a public method of a class - in complete isolation - by stubbing or mocking all its dependencies. When I refer to integration tests I mean testing classes that collaborate together. When I refer to acceptance tests I mean end-to-end tests.

Last Friday I sent a code review (what a wonderful invention the code review) to a friend and colleague, let’s call him D… hmm… Dumbledore! Yes! That will do. I sent a code review to Dumbledore and continued coding away. After a brief period of time Dumbledore dilligently went through my code and sent me the review back. I impatiently opened it and read:

It looks like you are testing an implementation detail.

Which started a great conversation about programming in general (what a wonderful invention the code review) and about testing behavior vs testing implementation in particular. I kept thinking about the testing behavior not implementation conundrum after work which lead me to write an email the day after and this blog post today. I think that some times I need time to think before I can articulate a good answer to stuff. Particularly with some techniques and practices that I have been doing for a while and I have internalized so much that I almost don’t question any more.

Here is what I concluded after some thought. The email read like this (foreground slowly fades into the past, the screen now in fuzzy shades of gray, Gmail):

I think that indeed the most ideal objective when testing is to test behavior, that’s what you really want to focus on when writing automated tests because it is the only real/final type of test that is going to tell you whether or not your application behaves in the way you want. But that means that, to really test behavior in complex systems you are going to need to write acceptance/integration tests (black-box tests), that is, expensive tests (expensive to setup and expensive to run). For instance, in the particular case that we were talking about in the code review, in order to write a real test of behavior I would have needed to write a black box test that would have tested the service, the inbox manager, any other inbox manager collaborators and probably a repository, NHibernate and the database. This is a huge investment and that’s why I usually like to write unit tests (test unit of code in isolation) instead when I am developing, particularly if I feel that time is running out XD (although in the best case scenario I would write an acceptance test and then TDD my way into a working implementation).

It is quite difficult to test behavior when writing a unit test for a complex class that has many collaborators, that in turn have more collaborators, that have in turn yet more collaborators. For instance,

  1. If you have a calculator with an add method that takes a couple of integers, then no problem, you can easily test behavior.
  2. If you have a console app that takes string arguments and adds them (string calculator kata?), you may have a class that does the arithmetic operations and an argument parser. You can test both individually for behavior and the whole console app with black box testing. Still no problem.
  3. If you have a huge application with tons of classes that depend on each other then testing gets harder. Black box testing becomes hard to setup and slow to run, so you lose one of the most useful/important characteristics of automated testing: short feedback loops. Something that quickly tells you that your code is [not] working as expected, and does it fast.

In case number 3, the case we find ourselves at. You have classes with tons of nested collaborators. You can, and should write acceptance tests, but the feedback loop for these tests is soooooooo slow. You write them, build, start backend, “ups, I made a stupid error”, fix, build, start backend, dammit, another error, and so on and so forth… that’s why, testing with acceptace tests is painful, and that’s where the value of unit tests lie. I can write a test, red, write code, green, refactor, green, write test, red, write code, green, refactor, green and so on super fast so that when I am ready to run the acceptance test I am in a position where I am pretty sure that it will pass.

And this brings us to the problem, How can I write unit tests for complex systems so I can test behavior instead of an implementation?. And that is a tricky question that requires consideration. In the case we are discussing about, I very distinctly tested an implementation:

ShouldGetTheFoldersViaTheInboxManager_WhenCalled (or something like that) where I asserted that a collaborator’s method had been called.

I could make it more behavior-y (and that’s an improvement indeed) by changing its name and body:

ShouldGetTheFoldersIntheUsersInbox_WhenCalled where I would setup the inbox manager stub and test that the service returns the aforementioned folders.

This is an improvement indeed but it still has a problem. I am testing an implementation disguised as a behavior. Why? Because I have explicitly set up the stub. (This test is the equivalent to the mock assertion, I just changed the side of the coin, particularly in this case where the only thing the method does is delegate)

In order to really test behavior in this particular case I think we would need to do a couple of things:

  1. Answer the question: What part of my code is just an implementation? How many collaborators and how deep are just part of this implementation detail? That would probably end up in us dividing the application in additional submodules with a more strict separation (which would be a good thing). We usually achieve this through projects/assemblies/layers, but it would be probably interesting to make use of more private classes and start thinking more deeply about when we use DI (dependency inversion) and IoC (inversion of control) to make an explicit separation between implementation details and the contract/service a given module fulfills/provides.
  2. Since we are using IoC heavily, we would need to improve our testing development environment to be able to inject all the dependencies that are part of the implementation, so that you won’t need to do that manually, and thus save time and pain.

Those are the things I can think about when reflecting about this problem, but there is a lot of people that knows more about this than I do that have much more to say.

Another different question is, is there value in unit tests that test an implementation? or is there value in testing units of code in isolation (because I think that most cases they are the same thing)?

The pros that I see in writing unit tests are the ones that I have always seen:

  • test ensures the code behaves as I want it to behave
  • fast to write, fast to run, short feedback loop, good value for price
  • find bugs with super-high granularity (if a unit test shows a bug, you pretty much know which class it is in, you don’t need to go throw all classes nor debug it, that’s priceless)
  • it forces you to write very simple classes and increase modularity and separation of concerns (because writing unit tests for fat, complex classes with tons of collaborators is horrible. We have a natural tendency to avoid pain)
  • increases developer confidence to refactor
  • documentation for my future self and other developers
  • if something changes that affects the unit under test, you will know it (regardless if it is a bug or a new implementation xD)

The cons are the ones that I have learnt to live with and that have made me reflect on improving the way I test:

  • they can be brittle in the sense that changes in the implementation may break your tests
  • they are not as trustworthy as higher level tests. You can make assumptions when stubbing collaborators that do not match reality and end up testing… something different

For instance, in the case of the calculator console app, you could have a calculator class that would call an argument parser to parse the arguments and then call the arithmetic unit to perform the arithmetics on those parameters. You could write a test to test that the calculator behaves in that way. GivenTwoNumbers_ShouldReturnItsSum and you would setup two stubs to do the parsing and the arithmetic calculation. The calculator class itself doesn’t do much, it justs delegates argument parsing to an argument parser class and arithmetic calculations to the other class. If you write a test to verify that the calculator behaves in that way, you are pretty much testing that algorithm: 1) parse 2)arithmetics, but more explicitely, there’s some IArgumentParser that is going to parse the arguments and some IArithmeticsUnit that is going to add them. What do you gain by writing this test? this particular test is of marginal value if you compare it to the unit tests for the arithmetic and the parser classes but you still gain something: 1) you know that the class works in the way you want, without the test, how would you know it works at all? 2) If you run the app and there’s a bug, you are going to be pretty sure that the bug is not in the calculator but in other dependencies.

And that’s pretty much it… I have to think about other specific examples within our application MagicFlow …

hmm… this could be a blog post xDDD

(foreground distorts into reality, welcome back to the present) And that was the email. Hope you found it interesting and feel free to comment and share your thoughs about unit testing and TDD!

I have been thinking for a while about writing more articles regarding my experience and journey with unit testing and TDD, so expect more of them coming. And particularly at this point where I am starting to question the definition of unit tests in isolation I learnt from the art of unit testing. All hope is not lost, I was just a liiiiitle bit slower than Roy :)

I used to feel that a ‘unit’ was the smallest possible part of a code base (a method, really). But in the past couple of years I’ve changed my mind. Here’s how I define a unit test, as of October 2011:

A unit test is an automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behavior of that unit of work.

A unit of work is a single logical functional use case in the system that can be invoked by some public interface (in most cases). A unit of work can span a single method, a whole class or multiple classes working together to achieve one single logical purpose that can be verified.

Some Interesting references

5 things that I like about codealike and 2 that I don't

Oh yesss… I just dit it… I just wrote a 5 things X blogpost… God save my soul.

Ok, now that I’ve got your attention (this 5 things X blog post titles never fail in that) I will continue writing a little bit more about how my experience with codealike has been so far. This article was actually meant to be part of the Boost Your Productivity With Codealike Insights review but, for once, I had the good judgement to show it to my girlfriend Malin first who wisely advised me to split it appart “I got bored after the 8th paragraph Jaime” - She said, and She was right… as usual (you may be nodding at this point XD).

1. Once and for All I have Irrefutable Proof That I Love Coding

Yes, you read it right. Once and for all I have irrefutable, specific, real proof that I love coding. Check this out. This right here is my productivity pattern per machine.

productivity pattern per machine in codealike

From it we can reach to several conclusions:

  • You now know my machine’s name, I may have to kill you
  • I wake up at 5 a.m. and code
  • Looks like sometimes I stay and work too late :(
  • Either my productivity sucks the whole day OR I am unusually uber-productive between 6 and 7 because I want to go home… so that I can… cuddle with Malin (…and code apparently)

2. Each Debug Session is a Failure

As a deep believer and practitioner of TDD and a member of the league of extraordinary gentle-unit-testers debugging is a failure. Every time I have to click that F5 and go into debug mode is a reminder of that moment of weakness when I decided not write a unit test, or that time I let a teeny tiny corner of the source code rot just enough… This month I failed 26 times. But next month will be better, one test/refactoring at a time, hell yeah.

Debugging is bad

3. You Can Drill Down to the Minutest Detail

Here it is. You are looking at how much time I spent coding on that javascript file (and at the fact that I do not seem to know how to spell Web XD). I have no idea about the utility of this feature, aside from the fact that long time reading within a method could indicate poor factoring and readability in that method, but it sure is a cool feature.

Codealike minutest detail

Oh, by the by, in C# code you can drill down to methods and not only see files.

4. You Can Healthily Compete With Your Teammates

Here you can see how this week is going so far. Daniel is kicking my butt, I have coded almost nil this week. That doesn’t necessarily mean I haven’t contributed to the team… I am funny:

Daniel vs Jaime on Codealike

And wait… yes, picture this. In a not too distant future codealike could add a gaming layer. You and your teammates would be able play games with each other based on the data in codealike, like… who’s the first to code 40 hours this month or…

5. My Coderbits Profile Is Getting Beefed Up

Yes, it is true, I must admit it, ever since I linked my coderbits account to codealike I became one of the top 200 devs.

codealike and coderbits

6. You can Get a Nice Summary Of The Languages You Use The Most

Look at this nice profile called Your Facts:

my facts

Ok, and that was it for the awesome stuff. Enough of the good cop. Time for some harshness and criticism.

-1. How the Heck Do I Change my Profile Picture?

No… for real… how do I change my profile picture?

-2. I Need More Education

Once thing that you will start feeling after using codealike for a short while is confusion (and curiosity XD). Particularly, you will ask yourself, where do these numbers come from? Why does it look like that I am working so few? Does the plugin fail to connect to the backend and some data is missed? Do I really work so few? But it cannot be…

So it would be pretty great if the guys at codealike could devise a way to educate us, their users, to better understand the data. And additionally, if they could, of course still leave the capability of being able to drill into the data, but dumb it down and provide more aggregation, tailored advice and more specific easy-to-understand KPIs. That would be great.

Well, that’s all for now. I was a little bit all over the place but I hope you liked it (and secretly wish that you had a laugh or two) :). Have a good one.

P.S. I just found out that there’s a Windows Phone Codealike app! Check that out…