Introduction to Unit Testing Part 3: Unit Testing Sucks

I don’t know about you, but I remember desperately wanting to be able to drive right up until I was fifteen years old and I got my learner’s permit. I thought about it a lot–how fun it would be, how much freedom I would have, how my trusty old bike would probably get rusty from disuse. About a month after getting my permit, I desperately wanted my license and to drive on my own without supervision. But I’m omitting a month there, during which an unexpected thing happened. I realized that driving was stupid and awful and it sucked and I hated it and I’d never do it, so just forget it!

It was in that month that the abstraction of operating a car and having freedom became the reality of hitting the gas when I meant to hit the brake pulling out of my driveway or not knowing when I was supposed to go after stopping at a stop sign. It was a weird mix of frustration, anger, and fear that tends to accompany new activities–even ones that you know will benefit you. And that’s why the title of this post isn’t simple link bait. I did that not to satirize a position, but to empathize. Like many things when you’re new to them, starting to unit testing quite frankly sucks. It’s frustrating, foreign, and hard to get right. Accordingly, it’s easy to abandon it when you have deadlines to meet.

Accident

This post is about minimizing frustration and barriers to adoption by staying focused and setting reasonable expectations. I would argue that if you’re new to writing tests, writing a few and enjoying localized success without high coverage is a lot more important than suddenly becoming a TDD (Test-Driven Development) expert with 100% test coverage right out of the gate (or at least trying to become one). Incremental progress is good.

Don’t Try TDD Just Yet

I’m a little torn as I write this, but the first thing that I’ll suggest is that you not try TDD if you have no experience unit testing. Some might disagree with this suggestion, but I think that you’re going to be trying to learn too many new things all at once and will be a lot more likely to get frustrated. Unit tests are simply pieces of code that you write, as covered in more detail in the last post in the series. It’s a new kind of code to be writing, but you’re just learning about new methods to call and attributes (or annotations, in Java) to use. You’ll get there.

But TDD is an entirely new way of writing code. It’s a discipline in which you do not write any production code until you have written a unit test that fails. Then you get that test and all other tests to pass and refactor the code as needed. Does that sound crazy (if you discount the fact that a number of developers you respect probably do it)? Exactly. Probably not for you right now. It’s a bridge too far, and you’re more likely to throw up your arms in disgust and quit if you try to learn both things right now. I speak from experience, as, years ago, I was introduced to unit testing and TDD at the same time. I was overwhelmed until I just went back to figuring out the whole unit testing thing alone first. Maybe that wouldn’t happen to you, but I’d caution you to be wary of learning these two things simultaneously.

So let’s stick to learning what unit tests are and how to write them.

Test New Classes Only

In my pluralsight course, I use the example of a method that identifies numbers as prime or not, and in a series of posts I did last fall on TDD, I use the example of something that calculates a bowling score. I’ve also done other code katas and exercises like these in the past to show people both the mechanics of unit testing and TDD.

When I do this, one of the things people frequently say is something along the lines of “pff…sure, when you’re writing something stupid and easy like a prime number finder, but there’s no way that would work on our code base.” I then surprise these people by agreeing with them. I’m sure it wouldn’t work on your code base. Why? Well, because unit tests don’t just magically spring up like mushrooms after a few days of rain. They’re more like roses–you have to plan for them from the start and carefully cultivate an environment in which they can thrive.

Some years back, I saw an excellent talk on “The Deep Synergy Between Testability and Good Design,” by Michael Feathers. I highly suggest watching this talk if you haven’t seen it, but to summarize, he states (and I agree) that well-designed and factored code goes hand in hand with testability. You’re much more likely to find that code written to be testable is good code and, conversely, code written without unit tests in mind is not the greatest. And so if you’re deeply invested in a code base that has never been covered by unit tests, it doesn’t surprise me to hear that you don’t think unit testing would work on your code. I imagine it wouldn’t.

But don’t throw out unit testing because it looks like it wouldn’t work in your code base. Just resolve to do it on new classes that you create. As you go along and get better at unit testing, you’ll start to understand how to write testable classes. It will thus get easier and easier to test all new additions to the code, and you’ll start to get the hang of it with relatively minimal impact on your existing code, your process, or your time. Starting to unit test doesn’t mean that you’re suddenly responsible for testing every line of code in history, nor does it mean you must test every single new line. Just start out by writing a few that you think will help.

Test Existing Code by Extracting Little Classes

Once you get the feel for adding unit tests for new classes/code that you add to the code base, it’s a good time to start taking baby steps toward getting tests in place for your legacy (non-tested) code. Now, some procedural, monolithic mass of code that wasn’t testable a month ago when you started out isn’t magically testable now because you have some practice. It’s still a problem.

You’re going to have to chip away at it. And you’re going to have to do this by developing a new skill: identifying pieces of functionality that you can pull out into new classes and test. Go look through methods and classes and find things that don’t have a lot of dependencies on class fields or (yuck) global/static variables. Excellent candidates for this are methods with pure in-memory operations and ones that deal largely with primitives. Do you have some gigantic method that has a whole region buried in it that does nothing but cobble together a string to be used later in the method? Pull that out into a new class, and write unit tests that make assertions about the string it returns.

As you practice this, you’ll get a better and better feel for what you can pull out with a minimum of friction. You’ll find yourself not only getting more of your codebase under test, but also that you’re improving its design and modularity.

Know When to Fold ‘Em

This is another one that’s hard to type, but you really have to learn to look at code and just say, “nope, not happening.” There are classes and methods that you simply are not going to be able to test unless you come back with a green belt in unit testing–or pair with someone who has hers. And, even then, the prognosis may be that you need to rewrite the legacy class/method altogether to make it testable. Here is a quick list of things that, early in your unit-testing career, you should consider to be deal-breakers and simply move on from to avoid frustration. As a beginner, avoid testing code (class methods and properties) that:

  1. Calls static methods. At best, a static method is functional and returns something that depends only on its inputs. If this is the case (such at functions like Math.Pow() or Math.Abs()), the code is still testable, but a far more common case, especially if the static methods are ones in your own code base, is that they manipulate some kind of global state. Global state is testability kryptonite. I’ll explain more later, but for now, please take my word for it.
  2. Invokes singletons. The singleton design pattern is used almost universally as a politically correct way to hide your global variables in plain sight. For what this means to testability, see the last bullet. If it calls singletons, forget it, move on.
  3. Dispatches background workers or manages threading. When unit tests are run, the unit test runner is responsible for managing threading and it will run your tests in parallel. If you’re trying to make sure your threads and thread management are in one state for production and another for testing, you are about to ruin your day and probably your week. It’s not worth it–don’t try.
  4. Accesses files, connects to databases, calls web services, etc. I mentioned this in the first post in the series, but that was in the context of saying that these things aren’t considered unit tests. Well, another issue here is that they’re also relatively brittle and long running. If you write tests that do these things, they’re going to fail at weird times and in unpredictable ways. You’ll be used to all of your tests passing and suddenly one fails and then passes again, and it turns out it’s because Bill from accounting bumped into the database server and its Nic card is a little “tricky.” If you have unit tests that fail for borderline-inconceivable reasons beyond your control, you will become discouraged.
  5. Code that triggers any of the above anywhere in the call stack. You don’t escape the problems of threading, global state, or externalities by not using them directly. If you trigger them, it’s the same difference.
  6. Classes that require crazy amounts of instantiation. If you want to test a method, but it has forty-five parameters, most of which are classes that are difficult or complex to create, forget it. That code sorely needs reworking, and creating massive, brittle tests for it this early in your career will be a world of pain. Chip away at making the design better before you tackle it.

Don’t worry–I’m not suggesting that you give up on a long timeline, and I’ll continue on with this series and discuss strategy for addressing these things later. But for now, just consider them signals that this code is out of bounds for testing. If you don’t, there’s a high likelihood that you’ll spin your wheels and get angry, frustrated, and irritable, making it more likely that you’ll give up. I can’t eliminate the frustration of being new at something like driving, but I can at least steer you away from six-way traffic lights and three-lane roundabouts.

  • Eric Olsson

    Excellent article, Erik.
    I agree–the TDD-er in me did cringe when reading not to bite off too much when first learning, but I believe that is wise advice.
    In addition, the list of “icebergs” which may sink the SS Unit Tester at the end is fantastic. I can echo the pains of unit tests involving multithreading. I have used a combination of event handling + AutoResetEvents to hack together some working tests in some cases, but it is a fragile solution. If you are working in WPF stuff, and you have to deliberately dispatch work on to the UI thread, that becomes troublesome in unit tests as well.
    The best advice I can offer is to try to unit test either side of the thread “boundary” and rely on integration tests or even SQA to find multithreading problems.

  • Adam

    Excellent article, really enjoying the series.

  • http://www.daedtech.com/blog Erik Dietrich

    I like how you describe the idea of threading “boundaries.” That makes a lot of sense to me. When it comes to concurrent programming, I like to try to abstract the dispatching to somewhere orthogonal to any important logic so those concerns are separated.

  • http://www.daedtech.com/blog Erik Dietrich

    I’m glad you’re liking it, and thanks for reading!

  • Pingback: The Morning Brew - Chris Alcock » The Morning Brew #1382()

  • Kris McGinnes

    Great article! You hit on all the pain points most people run into getting started with unit testing (including me). Tackling the problem with baby steps is by far the best way to go about introducing testing to a legacy code base.

  • Pingback: Introduction to Unit Testing Part 4: Design New Code For Testability | DaedTech()

  • http://www.daedtech.com/blog Erik Dietrich

    I’m glad you liked it. This whole series is a focus on the exact perspective that you mention since I’m trying to teach/convince at the same time in my day-to-day work.

  • Matt

    Hi, cool post.

    Challenge: Since it says “posted in Language Agnostic”, I challenge you to re-write the post without using the words “class” or “method” once :)

    Looking forward to finding the first and second parts of the series, thanks!

  • http://www.daedtech.com/blog Erik Dietrich

    I’m thinking it might be easier to change the tag to “OOP Language Agnostic.” :) Though, I’d love to have some time to play with the concepts involved in dynamic languages or using other paradigms.

    Anyway, glad if you’re enjoying the series — thanks for the comment.

  • serguei_p

    I am not sure you it is correct that static methods or singltons a generally difficult to write tests for.
    If the static methods are used where they supposed to be used (i.e. it is functional and return result depending on input parameters) and are not long-running, it is as easy to test the code that calls them as any other code.

  • http://www.daedtech.com/blog Erik Dietrich

    Well, I’d omit singletons from your line of argument. In essence, it sounds as though you’re saying that if you make sure to call static methods that are known to be simple and stateless, then you can easily test them (in the C# world, I’m picturing the math library, for instance). One can’t make that argument for singletons, since they are, by design, stateful — singletons are wrappers for your application’s global variables and thus not candidates for being “functional.” I’d unequivocally say that testing newbies should avoid singletons like the plague (or better yet, everyone should avoid writing them in the first place like the plague).

    Static methods, if stateless and simple, don’t really present a unit testing challenge, per se, other than the fact that you’re necessarily testing those methods and their functionality in addition to your own (strictly speaking, this becomes an integration test). But you can’t opt out of their functionality, so if a static method is long-running, as you mention, or if it contains state, or if it contains threading logic, or if it writes to disk or if it does any other huge host of things, you have no control over that when testing. And, worse, if someone ever changes it at any point in the future to do those things, you’re hosed in the future, unexpectedly. Statics introduce an externality beyond your control to your unit test. And, while Math.Log() is unlikely to write to a file or change functionality any time soon, other statics may, and I’ve witnessed firsthand the utter frustration and angst that this issue causes for newbies to testing.

    So, it may be overkill as advice, but I’d say it’s better to err on the side of caution than the side of frustration if you want to encourage the practice.

  • serguei_p

    My problem with this is the state of mind when it is seen to be OK when the design is compromised for the ease of testing.
    Singleton (as a state that is shared across the application) is natural. When someone instead creates a class that is then wired in Spring, he effectively creates the very same shared state by in a more convoluted way.

    Sometimes I feel that TDD makes rules not because they make sense but because the available tools are not good enough. At the end of the day we already have tools that allow to mock static methods (at least in Java).

    As for division between unit and integration testing, it is a bit stupid – should we start mocking Collections because otherwise our tests are integration tests?

    And don’t let me even start of the statement that “TDD leads to better design”. Yes, it leads to a better design if we are talking about developers who used to create messy and difficult to support code before embracing TDD (as a side-effect).
    The same as crutches help some people to move around (but you would not see healthy people using crutches for this purpose).

  • http://www.daedtech.com/blog Erik Dietrich

    I mostly don’t use the kinds of tools that allow you to mock statics/globals, except for mocking out file I/O and things like that (several such tools exist in the .NET world and I’ve used one in Java thats name escapes me since it’s been a while). So that hadn’t occurred to me, and it’s a fair point that you could use such a tool to mock out a static method call and then truly have a unit test. I wouldn’t advocate this approach, myself, because you’re still surrendering all of the maintenance benefits of polymorphism/IoC, but that doesn’t alter the fact that you’re right, and the option does exist.

    The idea of unit versus integration test to me is simply one of whether or not I can isolate the inputs to the behavior of the method I’m testing. But that doesn’t mean that I’d advocate achieving isolation by any means necessary. And I also have no problem with integration tests — I write those too — I’m just not writing a series of posts about them.

    As to the rest, /shrug/. Maybe I’m misunderstanding, but it sounds like you like singletons and don’t like TDD/inversion of control. I guess we’ll have to agree to disagree on both counts.

  • serguei_p

    I don’t like when something becomes the next fashionable dogma for all the wrong reasons.

    Yes, testing is important. No, one should not compromise the design to make the testing easier. Yes, some people would do stupid things with static methods and the static state. No, it does not mean that all static methods are necessary bad.

  • http://www.daedtech.com/blog Erik Dietrich

    I definitely agree that doing something simply because it seems like the thing to do is a misguided approach. Personally, I’m always operating under the assumption that there’s some better approach out there than the one I’m taking currently and I try to keep my eyes open for it.

  • Pingback: Intro to Unit Testing 7: Overcoming Inertia and Objections | DaedTech()

  • Lucas Manzke

    I have to agree with Erik here. I started TDD only a few days ago, but I already feel quite at home. To be honest, I had written an Android application that I wanted to write tests for, and stumbled upon the very obstacles that are mentioned in this article. First and foremost: Singletons. The thing with them is that I’d rather want to mock a whole object than a static method (which I think is what you were suggesting), with something like Mockito. Now, after rewriting the Android App with (Robo)Guice, I do not have to worry about that Singleton anymore, and even if i.e. my AuthController (keeping track of the Auth Status the whole time) is a shared instance across all classes, I can easily mock it out with Guice and test that the interactions with it are correct. I had a much harder time doing that when using Singletons and mocking these out somehow feels wrong.