Test Readability: Best of All Worlds

When it comes to writing tests, I’ve been on sort of a mild, ongoing quest to increase readability. Generally speaking, I follow a pattern of setup, action, verification in all tests. I’ve seen this called other things: given-when-then, etc. But when describing the basic nature of unit tests (especially as compared to integration tests) to people, I explain it by saying “you set the stage, poke it, and see if what happens is what you thought would happen.” This rather inelegant description really captures the spirit of unit testing and why asserts per unit test probably ought to be capped at one as opposed to the common sentiment among first time test writers, often expressed by numbering the tests and having dozens of asserts intermixed with executing code:

I think that was actually the name of a test I saw once: Test_All_The_Things(). I don’t recall whether it included an excited cartoon guy. Point is, that’s sort of the natural desire of the unit testing initiate — big, monolithic tests that are really designed to be end-to-end integration kinds of things where they want to tell in one giant method whether or not everything’s okay. From there, a natural progression occurs toward readability and even requirements documentation.

In my own personal journey, I’ll pick up further along that path. For a long time, my test code was always a monument to isolation, historically. Each method in the test class would handle all of its own setup logic and there would be no common, shared state among the tests. You could pack up the class under test (CUT) and the test method, ship them to Pluto and they would still work perfectly, assuming Pluto had the right version of the .NET runtime. For instance:

[TestClass]
public class MyTestClass
{
    [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
    public void Do_Something_Returns_True()
    {
        var classUnderTest = new ClassUnderTest(); //Setup

        bool actionResult = classUnderTest.DoSomething(); //Poke
            
        Assert.IsTrue(actionResult);  //Verify
    }
}

There are opportunities for optimization though, and I took them. A long time back I read a blog post (I would link if I remember whose) that inspired me to change the structure a little. The test above looks fine, but what happens when you have 10 or 20 tests that verify behaviors of DoSomething() in different circumstances? You wind up with a region and a lot of tests that start with Do_Something. So, I optimized my layout:

[TestClass]
public class MyTestClass
{
    [TestClass]
    public class DoSomething
    {
        [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
        public void Returns_True()
        {
            var classUnderTest = new ClassUnderTest(); //Setup

            bool actionResult = classUnderTest.DoSomething(); //Poke

            Assert.IsTrue(actionResult);  //Verify
        }

        [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
        public void Returns_False_When_Really_Is_False()
        {
            var classUnderTest = new ClassUnderTest() { Really = false }; //Setup

            bool actionResult = classUnderTest.DoSomething(); //Poke

            Assert.IsFalse(actionResult);  //Verify
        }
    }
}

Now you get rid of regioning, which is a plus in my book, and you still have collapsible areas of the code on which you can focus. In addition, you no longer need to redundantly type the name of the code element that you’re exercising in each test method name. A final advantage is that similar tests are naturally organized together making it easier to, say, hunt down and blow away all tests if you remove a method. That’s all well and good, but it fit poorly with another practice that I liked, which was defining a single point of construction for a class under test:

[TestClass]
public class MyTestClass
{
    private ClassUnderTest BuildCut(bool really = false)
    {
        return new ClassUnderTest() { Really = really };
    }

    [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
    public void Returns_True()
    {
        var classUnderTest = BuildCut(); //Setup

        bool actionResult = classUnderTest.DoSomething(); //Poke

        Assert.IsTrue(actionResult);  //Verify
    }

    [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
    public void Returns_False_When_Really_Is_False()
    {
        var classUnderTest = BuildCut(false); //Setup

        bool actionResult = classUnderTest.DoSomething(); //Poke

        Assert.IsFalse(actionResult);  //Verify
    }
}

Now, if we decide to add a constructor parameter to our class as we’re doing TDD, it’s a simple change in on place. However, you’ll notice that I got rid of the nested test classes. The reason for that is there’s now a scoping issue — if I want all tests of this class to have access, I have to put it in the outer class, elevate its visibility, and access it by calling MyTestClass.BuildCut(). And for a while, I did that.

But more recently, I had been sold on making tests even more readable by having a simple property called Target that all of the test classes could use. I had always shied away from this because of seeing people who would do horrible, ghastly things in test class state in vain attempts to force the unit test runner to execute their tests sequentially so that some unholy Singleton somewhere would be appeased with blood sacrifice. I tossed the baby with the bathwater — I was too hasty. Look how nicely this cleans up:

[TestClass]
public class MyTestClass
{
    private ClassUnderTest Target { get; set; }

    [TestInitialize]
    public void BeforeEachTest()
    {
        Target = new ClassUnderTest();
    }

    [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
    public void Returns_True()
    {
        //Setup is no longer necessary!

        bool actionResult = Target.DoSomething(); //Poke

        Assert.IsTrue(actionResult);  //Verify
    }

    [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
    public void Returns_False_When_Really_Is_False()
    {
        Target.Really = false; //Setup

        bool actionResult = Target.DoSomething(); //Poke

        Assert.IsFalse(actionResult);  //Verify
    }
}

Instantiating the CUT, even when abstracted into a method, is really just noise. After doing this for a few days, I never looked back. You really could condense the first test down to a single line, provided everyone agrees on the convention that Target will return a minimally initialized instance of the CUT at the start of each test method. If you need access to constructor-injected dependencies, you can expose those as properties as well and manipulate them as needed.

But we’ve now lost all the nesting progress. Let me tell you, you can try, but things get weird when you try to define the test initialize method in the outer class. What I mean by “weird” is that I couldn’t get it to work and eventually abandoned trying in favor of my eventual solution:

[TestClass]
public class MyTestClass
{
    protected ClassUnderTest Target { get; set; }

    [TestInitialize]
    public void BeforeEachTest()
    {
        Target = new ClassUnderTest();
    }

    [TestClass]
    public class DoSomething : MyTestClass
    {
        [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
        public void Returns_True()
        {
            //Setup is no longer necessary!

            bool actionResult = Target.DoSomething(); //Poke

            Assert.IsTrue(actionResult);  //Verify
        }

        [TestMethod, Owner("ebd"), TestCategory("Proven"), TestCategory("Unit")]
        public void Returns_False_When_Really_Is_False()
        {
            Target.Really = false; //Setup

            bool actionResult = Target.DoSomething(); //Poke

            Assert.IsFalse(actionResult);  //Verify
        }
    }
}

So at the moment, that is my unit test writing approach in .NET. I have not yet incorporated that refinement into my Java work, so I may post later if that turns out to have substantial differences for any reason. This is by no means a one size fits all approach. I realize that there are as many different schemes for writing tests as test writers, but if you like some or all of the organization here, by all means, use the parts that you like in good health.

Cheers!

  • http://www.facebook.com/profile.php?id=669328712 Chuck Bryan

    I like your SPV arrangement! I use an abstract base “template” class that I call AAA. It has an “TestInitalize” method that calls two abstract methods (Arrange, Assert) that you implement in each implementing class. Then, the “Assert” part is just a regular [Test]. This drives the arrangement of your tests around the “context” established in your Arrange method and by the time you get to the actual test, you have already performed the test in the Act. I tend to have multiple tests in each class and LOTS of classes (not sure if that is good).

  • Eric Olsson

    Nice post, Erik. You’ve forced me to once again face my prejudices. :) I’ve always shied away from instantiating my class-under-test in a setup/initialize method, but I believe it is because I have seen horrible things done in the setup/initialize methods that cause side effects once a 2nd author begins to add tests to a test class. I would always tend to your approach wherein my test could be shot to Pluto & run perfectly fine since all is self-contained.

    However, I recognized the needless repetition that my approach incurred, so my answer was to define a class-level Func within the test class (by convention, I called it _constructor) which would essentially be a factory method. One nice feature that bought me was that if I wanted to test any logic in my constructor, I could set up my mock object first, perform the construction, and then examine the mock. Typically, this would come up when I wanted to ensure that a class was listening to an Event Aggregator (or other message bus-type mechanism).

    And your listing of the nested class style of test organization reminded me of a post I read earlier this year on Phil Haack’s blog. I was wondering if this is the place where you got it: http://haacked.com/archive/2012/01/01/structuring-unit-tests.aspx

  • http://www.daedtech.com/blog Erik Dietrich

    That’s a fascinating concept. I think there’s something fundamentally appealing about centering your testing around the same setup context as opposed to simply the same public member. I think I’ll have to try that out to see how it feels. Thanks for the the comment and the idea!

  • http://www.daedtech.com/blog Erik Dietrich

    That article by Phil Haack rings a bell — I’ve definitely read it. That could very well be where I got it.

    And that’s a good point about wanting to examine what happens in a mock object during constructor time. I’ve only theoretically addressed that with the idea that I could expose the mock objects as properties and inspect them as well, but it might be worth contemplating something that was a little less disjointed. Maybe you can show me a code sample of what you do some time. :)

  • Pingback: Scoping And Accessibility Quirks in C# | DaedTech