Ladies and gentlemen: The blog post you are about to read is based on a true story/e-mail. Only the names of classes and methods have been changed to protect the privacy and innocence of the source code. The facts remain unchanged and the contents unadultered…

Disclaimer: I love automated testing and I have been a practicioner of TDD since 2010.

When I refer to unit tests in this article I use the most strict definition of unit tests, that is, test a unit of code - a public method of a class - in complete isolation - by stubbing or mocking all its dependencies. When I refer to integration tests I mean testing classes that collaborate together. When I refer to acceptance tests I mean end-to-end tests.

Last Friday I sent a code review (what a wonderful invention the code review) to a friend and colleague, let’s call him D… hmm… Dumbledore! Yes! That will do. I sent a code review to Dumbledore and continued coding away. After a brief period of time Dumbledore dilligently went through my code and sent me the review back. I impatiently opened it and read:

It looks like you are testing an implementation detail.

Which started a great conversation about programming in general (what a wonderful invention the code review) and about testing behavior vs testing implementation in particular. I kept thinking about the testing behavior not implementation conundrum after work which lead me to write an email the day after and this blog post today. I think that some times I need time to think before I can articulate a good answer to stuff. Particularly with some techniques and practices that I have been doing for a while and I have internalized so much that I almost don’t question any more.

Here is what I concluded after some thought. The email read like this (foreground slowly fades into the past, the screen now in fuzzy shades of gray, Gmail):

I think that indeed the most ideal objective when testing is to test behavior, that’s what you really want to focus on when writing automated tests because it is the only real/final type of test that is going to tell you whether or not your application behaves in the way you want. But that means that, to really test behavior in complex systems you are going to need to write acceptance/integration tests (black-box tests), that is, expensive tests (expensive to setup and expensive to run). For instance, in the particular case that we were talking about in the code review, in order to write a real test of behavior I would have needed to write a black box test that would have tested the service, the inbox manager, any other inbox manager collaborators and probably a repository, NHibernate and the database. This is a huge investment and that’s why I usually like to write unit tests (test unit of code in isolation) instead when I am developing, particularly if I feel that time is running out XD (although in the best case scenario I would write an acceptance test and then TDD my way into a working implementation).

It is quite difficult to test behavior when writing a unit test for a complex class that has many collaborators, that in turn have more collaborators, that have in turn yet more collaborators. For instance,

1. If you have a calculator with an add method that takes a couple of integers, then no problem, you can easily test behavior.
2. If you have a console app that takes string arguments and adds them (string calculator kata?), you may have a class that does the arithmetic operations and an argument parser. You can test both individually for behavior and the whole console app with black box testing. Still no problem.
3. If you have a huge application with tons of classes that depend on each other then testing gets harder. Black box testing becomes hard to setup and slow to run, so you lose one of the most useful/important characteristics of automated testing: short feedback loops. Something that quickly tells you that your code is [not] working as expected, and does it fast.

In case number 3, the case we find ourselves at. You have classes with tons of nested collaborators. You can, and should write acceptance tests, but the feedback loop for these tests is soooooooo slow. You write them, build, start backend, “ups, I made a stupid error”, fix, build, start backend, dammit, another error, and so on and so forth… that’s why, testing with acceptace tests is painful, and that’s where the value of unit tests lie. I can write a test, red, write code, green, refactor, green, write test, red, write code, green, refactor, green and so on super fast so that when I am ready to run the acceptance test I am in a position where I am pretty sure that it will pass.

And this brings us to the problem, How can I write unit tests for complex systems so I can test behavior instead of an implementation?. And that is a tricky question that requires consideration. In the case we are discussing about, I very distinctly tested an implementation:

ShouldGetTheFoldersViaTheInboxManager_WhenCalled (or something like that) where I asserted that a collaborator’s method had been called.

I could make it more behavior-y (and that’s an improvement indeed) by changing its name and body:

ShouldGetTheFoldersIntheUsersInbox_WhenCalled where I would setup the inbox manager stub and test that the service returns the aforementioned folders.

This is an improvement indeed but it still has a problem. I am testing an implementation disguised as a behavior. Why? Because I have explicitly set up the stub. (This test is the equivalent to the mock assertion, I just changed the side of the coin, particularly in this case where the only thing the method does is delegate)

In order to really test behavior in this particular case I think we would need to do a couple of things:

1. Answer the question: What part of my code is just an implementation? How many collaborators and how deep are just part of this implementation detail? That would probably end up in us dividing the application in additional submodules with a more strict separation (which would be a good thing). We usually achieve this through projects/assemblies/layers, but it would be probably interesting to make use of more private classes and start thinking more deeply about when we use DI (dependency inversion) and IoC (inversion of control) to make an explicit separation between implementation details and the contract/service a given module fulfills/provides.
2. Since we are using IoC heavily, we would need to improve our testing development environment to be able to inject all the dependencies that are part of the implementation, so that you won’t need to do that manually, and thus save time and pain.

Those are the things I can think about when reflecting about this problem, but there is a lot of people that [knows more about this than I do](http://blog.ploeh.dk/tags.html#Unit Testing-ref) that have much more to say.

Another different question is, is there value in unit tests that test an implementation? or is there value in testing units of code in isolation (because I think that most cases they are the same thing)?

The pros that I see in writing unit tests are the ones that I have always seen:

• test ensures the code behaves as I want it to behave
• fast to write, fast to run, short feedback loop, good value for price
• find bugs with super-high granularity (if a unit test shows a bug, you pretty much know which class it is in, you don’t need to go throw all classes nor debug it, that’s priceless)
• it forces you to write very simple classes and increase modularity and separation of concerns (because writing unit tests for fat, complex classes with tons of collaborators is horrible. We have a natural tendency to avoid pain)
• increases developer confidence to refactor
• documentation for my future self and other developers
• if something changes that affects the unit under test, you will know it (regardless if it is a bug or a new implementation xD)

The cons are the ones that I have learnt to live with and that have made me reflect on improving the way I test:

• they can be brittle in the sense that changes in the implementation may break your tests
• they are not as trustworthy as higher level tests. You can make assumptions when stubbing collaborators that do not match reality and end up testing… something different

For instance, in the case of the calculator console app, you could have a calculator class that would call an argument parser to parse the arguments and then call the arithmetic unit to perform the arithmetics on those parameters. You could write a test to test that the calculator behaves in that way. GivenTwoNumbers_ShouldReturnItsSum and you would setup two stubs to do the parsing and the arithmetic calculation. The calculator class itself doesn’t do much, it justs delegates argument parsing to an argument parser class and arithmetic calculations to the other class. If you write a test to verify that the calculator behaves in that way, you are pretty much testing that algorithm: 1) parse 2)arithmetics, but more explicitely, there’s some IArgumentParser that is going to parse the arguments and some IArithmeticsUnit that is going to add them. What do you gain by writing this test? this particular test is of marginal value if you compare it to the unit tests for the arithmetic and the parser classes but you still gain something: 1) you know that the class works in the way you want, without the test, how would you know it works at all? 2) If you run the app and there’s a bug, you are going to be pretty sure that the bug is not in the calculator but in other dependencies.

And that’s pretty much it… I have to think about other specific examples within our application MagicFlow …

hmm… this could be a blog post xDDD

(foreground distorts into reality, welcome back to the present) And that was the email. Hope you found it interesting and feel free to comment and share your thoughs about unit testing and TDD!

I have been thinking for a while about writing more articles regarding my experience and journey with unit testing and TDD, so expect more of them coming. And particularly at this point where I am starting to question the definition of unit tests in isolation I learnt from the art of unit testing. All hope is not lost, I was just a liiiiitle bit slower than Roy :)

I used to feel that a ‘unit’ was the smallest possible part of a code base (a method, really). But in the past couple of years I’ve changed my mind. Here’s how I define a unit test, as of October 2011:

A unit test is an automated piece of code that invokes a unit of work in the system and then checks a single assumption about the behavior of that unit of work.

A unit of work is a single logical functional use case in the system that can be invoked by some public interface (in most cases). A unit of work can span a single method, a whole class or multiple classes working together to achieve one single logical purpose that can be verified.

Some Interesting references

• Ian Cooper talks about TDD and where did it all go wrong
• Steve Freedman talks about TDD (that’s now what we meant)
• [Mark Seemann on Unit Testing and TDD](http://blog.ploeh.dk/tags.html#Unit Testing-ref)
• Advanced Unit Testing
• Outside-In TDD

Oh yesss… I just dit it… I just wrote a 5 things X blogpost… God save my soul.

Ok, now that I’ve got your attention (this 5 things X blog post titles never fail in that) I will continue writing a little bit more about how my experience with codealike has been so far. This article was actually meant to be part of the Boost Your Productivity With Codealike Insights review but, for once, I had the good judgement to show it to my girlfriend Malin first who wisely advised me to split it appart “I got bored after the 8th paragraph Jaime” - She said, and She was right… as usual (you may be nodding at this point XD).

1. Once and for All I have Irrefutable Proof That I Love Coding

Yes, you read it right. Once and for all I have irrefutable, specific, real proof that I love coding. Check this out. This right here is my productivity pattern per machine.

From it we can reach to several conclusions:

• You now know my machine’s name, I may have to kill you
• I wake up at 5 a.m. and code
• Looks like sometimes I stay and work too late :(
• Either my productivity sucks the whole day OR I am unusually uber-productive between 6 and 7 because I want to go home… so that I can… cuddle with Malin (…and code apparently)

2. Each Debug Session is a Failure

As a deep believer and practitioner of TDD and a member of the league of extraordinary gentle-unit-testers debugging is a failure. Every time I have to click that F5 and go into debug mode is a reminder of that moment of weakness when I decided not write a unit test, or that time I let a teeny tiny corner of the source code rot just enough… This month I failed 26 times. But next month will be better, one test/refactoring at a time, hell yeah.

3. You Can Drill Down to the Minutest Detail

Here it is. You are looking at how much time I spent coding on that javascript file (and at the fact that I do not seem to know how to spell Web XD). I have no idea about the utility of this feature, aside from the fact that long time reading within a method could indicate poor factoring and readability in that method, but it sure is a cool feature.

Oh, by the by, in C# code you can drill down to methods and not only see files.

4. You Can Healthily Compete With Your Teammates

Here you can see how this week is going so far. Daniel is kicking my butt, I have coded almost nil this week. That doesn’t necessarily mean I haven’t contributed to the team… I am funny:

And wait… yes, picture this. In a not too distant future codealike could add a gaming layer. You and your teammates would be able play games with each other based on the data in codealike, like… who’s the first to code 40 hours this month or…

5. My Coderbits Profile Is Getting Beefed Up

Yes, it is true, I must admit it, ever since I linked my coderbits account to codealike I became one of the top 200 devs.

6. You can Get a Nice Summary Of The Languages You Use The Most

Look at this nice profile called Your Facts:

Ok, and that was it for the awesome stuff. Enough of the good cop. Time for some harshness and criticism.

-1. How the Heck Do I Change my Profile Picture?

No… for real… how do I change my profile picture?

-2. I Need More Education

Once thing that you will start feeling after using codealike for a short while is confusion (and curiosity XD). Particularly, you will ask yourself, where do these numbers come from? Why does it look like that I am working so few? Does the plugin fail to connect to the backend and some data is missed? Do I really work so few? But it cannot be…

So it would be pretty great if the guys at codealike could devise a way to educate us, their users, to better understand the data. And additionally, if they could, of course still leave the capability of being able to drill into the data, but dumb it down and provide more aggregation, tailored advice and more specific easy-to-understand KPIs. That would be great.

Well, that’s all for now. I was a little bit all over the place but I hope you liked it (and secretly wish that you had a laugh or two) :). Have a good one.

P.S. I just found out that there’s a Windows Phone Codealike app! Check that out…

The Barbaric Development Toolbox is a series of posts where I write about awesome and indispensable tools and libraries that enhance productivity and make our lives as software developers much easier and enjoyable.

I have seen very few people as crazy about productivity as we software developers. I have no idea where that tendency or behavior comes from, but the truth is that we spend countless hours learning every shortcut there is to improve our programming fu. We learn to master our IDEs and text editors, put up with the steep learning curve of vim, or the strange incantations of Emacs. We use tools like ReSharper, JustCode or CodeRush, and learn disciplines like GTD or the Pomodoro Technique. All and everything just to be able to do more, to be better, to master our craft.

What if I told you that there is a new super interesting, innovative and all-hands-down-just-cool tool that lets you monitor every second of every minute you spend in Visual Studio? What if I told you that you can see how much time you spend reading, coding, building and debugging? How much time you do either of these things within a solution, a project, a class and to the minutest detail: a method!? Or which hours of the day you are more productive at, or which languages you use most often? What would you do with this kind of information?!? Behold!! Because codealike is here and is bringing all these insights and more to your nearest Visual Studio!

codealike is a Visual Studio (and Eclipse) plugin that records enormous amounts of information on how you spend your time in Visual Studio, and then weaves it in wonderful and insightful ways so that you can take the most value out of it. With codealike you can look at your code and your work patterns in a total different way than ever before. You can divine the secret story of your code, because your code has a story to tell and, until now, you haven’t been listening.

But let’s be less fluffy and go down to the specifics. These are some cool things you can find out with codealike:

• How much time you have spent coding, debugging, building and outside of Visual Studio
• How often you were interrupted in a given period of time
• How many times you’ve been on fire (in the mythical flow)
• How focused you were during a period of time and in different machines
• See in which solutions, projects, files, classes and methods you spend the most time
• See which programming languages, frameworks you spend time using
• See a very detailed timeline of how and where you spent your day
• Analyze the nitty-gritty details on how you spend your time in every solution, project, file, class and method
• Rank and compare yourself to other codealike users
• Compare yourself with a version of you from another space and time (to see how you have improved as the time passes)
• Use it within your team and get a holistic view on how the whole team is doing.

Which is infinitely better put forward in this video:

If you want to start learning more about how you code, go to codealike.com and create a free account. You just need to sign-up, download the plugin and get rolling.

VIP Premium Licenses Limited Time Offer

I have a limited time offer brought to you directly by the codealike guys. I have 30 premium licenses for a whole year to give away to you and an unlimited number of 20% discounts. The only thing you need to do is to tweet and share this blog post and reach me on twitter or via mail for your reward. It would be beyond awesome if you could write a comment about what you think about codealike, your favorite feature, is there anything that you are missing? or whatever crosses your mind at that particular moment in time.

The licenses and discounts are only available until the 2nd of August. Be swift.

Parting Thoughts

I have been using RescueTime for a while (ever since I heard about it in Scott Halseman’s great talk on productivity), and when codealike came out I went bananas. What!? This is RescueTime for .NET developers!! A w e s o me! There’s no need to say that I started using it right away.

It was a great idea, but I was a little bit disappointed with the information it initially provided as it felt quite scarce and of little utility - I think that is how I remember it although my mind may be playing tricks on me. Anyhow, what I want to get to, is to the surprise I felt when, a month after I started using it, codealike had become ten times better. And so it has continued, getting better and better at an amazing speed.

Today codealike provides a lot of interesting and helpful information, but I cannot help to think about all the possibilities and usages of all that information that are yet to come, all the tailored advice based on your personal data. The sky is the limit guys. :) (nope, I couldn’t resist the smiley).

Appendix, or on Another Interesting Less Known Facts About Codealike and its Makers

Did you know that…

• codealike uses RavenDB on Azure and released the first version of codealike in 6 weeks?
• the codealike guys collected some interesting insights on how we developers code based on the information they collected last year on 2013?
• corvalius, the company behind codealike, is an Argentinian company with an awesome manifesto we should all learn from?

Dev Talk Monday is the series that brings you awesome dev talks every Monday

Last week I came upon this talk from dotJS 2012 completely by chance: What is Open Source and Why I Feel So Guilty?. In it, with tons of candor, sincerity and humor, @fat(bootstrap, ratchet, hogan, etc) narrates the history of open source from the closeness of the IBM days to GitHub and beyond, tells about the struggles of an open-source contributor, about burnout, puppies and makes a plea for the future to fucking revolt and fix this shit.

You can find the talk on YouTube together with all videos from the dotJS Conference.

The slides are freaking awesome too! - in a brilliant nerdy way. You can follow @fat on twitter and at his blog, he has some very nice - yet old - articles like this one, or this one or this other one… (voice slowly fading in the distance).

Hi!

I was preparing a demo for the upcoming post on Barbarian Meets Knockout.js and jsFiddle just died on me, so I thought I could make the best of it and fill in the gap with an update on what I have been working on during the past weeks and what I am planning to do in the future.

The last few months have been pretty exciting. I finally made the decision to start my own company on the side of my job at medius. Yey! As soon as I realized that the time had come - which is to say, the idea nearly made my head burst open after a slow fermentation process that had been ongoing for the past three years - I quickly did all the paperwork, created a basic starting website and got on hacking on quiz4couples. My first intention was to create a new version of the app for iOS but, after receiving many a signal from the Universe, I have changed my mind and I am going to focus my efforts instead of spreading myself thin. The plan will thus be to continue attempting to prove that the app is economically viable with a couple more iterations on the Windows Phone version I have today.

I am still a full-time employee at medius so it is pretty tough to find time to work on my company. I don’t know how other people manage but I, myself, have devised what I call - The Magic Week - which pretty much means, wake up early as hell so you can put 2-3 hours before you go to work - and reserve a couple of evenings a week to write in this blog and just learn new stuff. I also give myself a rating - with magic stars, true story - on how well I stick to the schedule every week and so far it’s been working pretty good… mental note: some time in the future I need to write about this weird productivity systems I follow… :)

Anyhow, it’s been pretty awesome so far, having complete ownership of the whole product development process, the whole decision-making is kind of… freeing, kind of liberating. You are free to roam in whichever direction you desire to your heart’s content, ain’t that something. It is true that some times it is hard to focus on the task at hand. But I have found that having three daily big wins, using the pomodoro technique, and maintaining an orderly, clutter-free man-cave are invaluable to keep me productive.

Ups. There goes the go-to-sleep-dude alarm in full swing. I’ll have to leave writing about other exciting things I am planning for the future, for another day.

Good night.