DaedTech

Stories about Software

By

You’re Doin’ It Wrong

I was chatting with someone the other day about some code that he had written, and he expressed that he wished he had done it another way. He said that there was a bunch of duplication and that, while he wasn’t sure what he should have done, he didn’t think it seemed quite right. I took a look at the code and was inclined to agree, though I could think of how I might have approached it.

This led me to think a little more broadly about writing code that you feel not quite right about. One might simply call this a “code smell”, but I think a distinction can be made. I personally associate code smells more with code from which you are somewhat removed, either because someone else wrote it or because you wrote it some time back. That is, you wouldn’t write code that “smelled” to you if it smelled as you were writing it (hopefully).

So, what I’m documenting here is not so much ways to look at code and “smell” that something is wrong, but rather a series of heuristics to keep in mind during development to know that you’re probably doing something wrong. Code smells are more macroscopic in nature with things like “parallel inheritance hierarchy” and “anemic domain model”. These are practices that, as I’ve learned from experience, seem to lead to bad code.

Typing the same thing again and again

The incident that made me think of this involved a piece of code that looked something like this:

Now, there are a few things going on here that could certainly be improved upon, but the first one that jumps out to me is that if you’re appending numbers to fields or properties in class, you’re probably doing something wrong. We could certainly take our IDs and names and turn them into arrays, or lists or collections and iterate over them. But, even doing that, you’d have a bunch of parallel loops:

Each place that you want to do something with customer IDs, you’ll also have to do it with names. This gets particularly ugly if you decide you want to sort the lists or something like that — now you have to make sure the lists are rearranged in order – you take on responsibility for guaranteeing that your multiple list implementation remains consistent. This is repetitive still. Better to create a customer model struct with properties name and ID or something, and expose a list (or observable collection or whatever) of that.

The heuristic I can offer here is that if you’re typing the same thing over and over again, you’re probably doing it wrong. This is an offshoot of the DRY (Don’t Repeat Yourself) principle, but ironically, I think it bears some repeating here. DRY doesn’t simply mean that you shouldn’t copy and paste code, which is what many people remember. It doesn’t even mean that you shouldn’t duplicate code. Dave Thomas calls it something “much grander”:

DRY says that every piece of system knowledge should have one authoritative, unambiguous representation. Every piece of knowledge in the development of something should have a single representation. A system’s knowledge is far broader than just its code. It refers to database schemas, test plans, the build system, even documentation.

There should be no duplication of knowledge in the project, period. As such, typing the same thing over and over again has the whiff of duplication, even if you aren’t copy/pasting. If you perform a series of operations on customer ID 1 and then customer name 1, and then customer ID 2, and then customer Name 2, etc, you’re repeating that operation over and over again, albeit with a nominally different target. But, what if in the database or lower layers of memory, I switch customers 1 and 2? The 1 operation is then done on 2 and vice versa. These operations are interchangeable – we have duplicated knowledge. And, duplicated knowledge eventually leads to things getting out of sync as the system evolves, which means maintenance headaches, code rot, and defects.

Making a private member public and/or an instance method static

Let’s say that we have the following classes:

Now, let’s say that the GUI is bound directly to the customer object, and that a new requirement comes in – we want to display customer stuff in black, unless most of the customers have been processed (say, 3/4s of them), in which case we want to display them in red — for all customers. Let’s also assume that we have some kind of value converter that converts a boolean to red/black for binding. Here’s how someone might accomplish this:

Don’t do that! That might be the quickest and dirtiest way to accomplish the task, but it sure is ugly. We break encapsulation by making an internal detail of the processor public, we create a global variable to get around the issue that the customer has no reference to the processor, and we’ve only created a half-way solution because the customer still has no mechanism for knowing when the processor’s count passes the magic number 9 – we’d need to expose a global event or callback so that the customer would know when to raise property changed on its color. This is also a concrete and circular coupling – CustomerProcessor and Customer now know all about one another and might as well be one static, global mashup.

Private members are private for a reason — they’re implementation details hidden from the code base at large. By having the magic number 12 and the processed count hidden in the processor, I can later change the scheme by which the processor evaluates “finished” without breaking anything else. But, the way I’ve mangled the code here, the requirement that “red” be 3/4s of the finished count is now leaked all over the place. If I go to the processor and decide that finished count is now 400 (or better yet, configurable and read from persistence), our new requirement is no longer satisfied, and as the maintenance programmer for CustomerProcessor, I have no way of knowing that (except maybe for the hint that someone exposed my private variable).

Likewise, instance members are instance members for a reason. If I later decide that I want two separate customer processors working in two separate screens, I’m now hosed with these changes. However many processors I create, if any of them process 12 records, they’re all finished.

Far too often, the reason for making privates public or instance members static is convenience and not wanting to think through dependencies. Both of those make the information that you want easier to get at. But, I submit that if this is your solution to get your objects to talk to one another, you’re likely creating a short-sighed and misguided solution. In our example above, exposing an “IsAlmostFinished()” instance member on the processor and then having the GUI bind to something that knows about the customers and their processor is a better solution. It requires a bit more planning, but you’ll be much happier in the long run.

You can’t describe a method in a simple sentence using its parameters and return value

“Too many parameters” is considered a code smell. But, how many is too many? Recently, I stumbled across a method that looked something like this:

This method doesn’t have too many parameters at all, I wouldn’t think. It has exactly one parameter, which is basically the programmatic equivalent of the ubiquitous “Are you sure you want to…” message box. But, I couldn’t for the life of me summarize it in a simple sentence, and perhaps not a paragraph. It was something like “If the user passes in true, then find the hardware processor and hardware presenter, and if each of those isn’t null, find the database manager, but if the hardware presenter is null, then create a file, and…”

If you’re writing a method, do a thought exercise. If someone stopped by and said, “whatcha doin?”, could you look up at them and quickly say “I’m writing a method that…”? Or would you say, “go away and don’t break my concentration, or I’ll never remember what this thing was supposed to do”? If it’s the latter, you’re doing it wrong.

We can only process so much information at once. If you’re reading through someone else’s code or your own code from a while ago, methods that are small and focused are easy to understand and process mentally. Methods that have dozens of parameters or do dozens of things are not. It doesn’t matter how smart you are or how good at programming you are – when you encounter something like that, you’ll put your pen to the monitor and start tracing things or take that same pen and start taking notes or drawing diagrams. If you have to do that to understand what a method does, that method sucks.

Consider the following interfaces:

There are no method definitions at all – just signatures. But, I’d be willing to be that you can already tell me exactly what will happen when I call those methods, and I bet you can tell me in a simple, compact sentence. Wouldn’t it be nice if, when you were coding, you only encountered methods like that? Well, if you think so, do others and your future self a favor, and only define and code them like that.

Defining class behavior for one specific client

Remember that method that I showed above? The one with the “are you sure” boolean parameter? I’m pretty sure that was neither the author’s idea of a joke nor was it quite as silly as it looked. I’d wager dollars to donuts that this is what happened:

When looked at in this light, it makes sense. There was some concept that a class was too large and that concerns should be separated. Taken as a gestalt, it makes sense, but simply looking at the extracted class, “DoSomething(bool)” looks pretty dumb.

While the effort to refactor to smaller units is laudable, creating an interface that only makes sense for one client is not so good. Your classes should make sense on their own, as units. If you extract a class that seems hopelessly coupled to another one, it’s probably time to reconsider the interface, or the “seam” between these two classes. In the case above, the boolean parameter need not exist. “IsSomethingTrue()” can exist in an if condition prior to invocation of the parameterless “DoSomething()” method. In general, if your class’s public interface doesn’t seem to make sense without another class, you’re probably doing it wrong. (This is frequently seen in “helper” classes, which I mentioned in this post about name smells).

By

Offering Constructive Criticism without Breaking the Creative Spirit

Wet Behind the Ears

As a child in secondary school, you are instructed to do work and have very little choice in the whys and hows of the matter. This tends to be true in college education as well, although as one progresses, some choices are left to the student. Generally, these are vetted heavily by professors or teaching assistants. This trend generally continues right on into the business world, perhaps even with a bit of regression. New, entry-level hires are usually given pretty specific instructions and trusted very little – all work is checked heavily in the majority of cases.

This comes to be commonplace enough over the course of a young life that you don’t think much of it. I know that when I came into the work force, I took it as a matter of course that this would happen and that, in the majority of cases, the people vetting my work and giving me instructions knew what they were doing and I didn’t. I was right about this.

This isn’t to say I didn’t have the childhood tendency to be constantly convinced that I was right about things. The cliche of your parents knowing nothing when you’re a high school kid and getting a lot smarter somehow over the next 10 year was true for me. But, this was really only the case in my personal life – in business and academia, I took it as a matter of course that those with more experience than me knew more than me and that I could learn from then. Again, I was right about this.

The Bulb Comes On A Little

After a while, an interesting thing started to happen. I gained enough experience and confidence after a bit of time in the workforce to actually form my own opinions. Instead of always taking it as a matter of course that reviewers and managers were right about everything, I’d start to think to myself, “I’m not sure that’s the best way to do it.” But, I’d do it anyway, and often this would result in me learning how wrong I had been. But, every now and then, it would result in my suspicions being corroborated, and I discovered that being the junior team member, person whose code was being reviewed, employee, etc, didn’t mean, ipso facto, that I was wrong and they were right.

I tend to learn from my peers very quickly and soak up knowledge from them like a sponge. Perhaps this is stubbornness and I don’t like being wrong, so I burn the midnight oil to fix my weaknesses. Perhaps it’s just that I have a good work ethic and lots of natural curiosity. Perhaps I’m a perfectionist. I don’t really know, and I don’t think it much matters. End result of this is that the times when I was right but told to do the wrong thing grew more frequent over the course of time, and that became frustrating to me. I wanted the benefit of the input I was getting, when that input was right, but not to be saddled with it when it was wrong.

The Wrong Approach

A cynical trick that I learned pretty quickly was that meetings to evaluate proposals or review work were generally of a fixed length duration. If I brought in a beautiful, well thought out proposal, some of my precious items would be critiqued, and I’d have to compromise on them. But, if I brought in the same proposal with a few obvious mistakes, the duration of the meeting would be occupied by suggestions to ‘fix’ my red herrings, and my proposal/design/code would remain intact as I had originally envisioned it.

I’m ashamed to say that this worked exactly as I intended at the time. Meetings went exactly as I planned, and I proceeded without any serious review of my concepts. I had, effectively, swung from naive assumption that others were always right to an equally naive assumption that I needed no input from anyone else. In the end, I made some mistakes that I had to put in long hours fixing and realized later that it was entirely possible someone could have anticipated the blunder that I had missed. Now that I’m too old to know everything, I understand exactly how silly this was.

How Did It Come To This?

Reflecting back on my hubris years later, I understand something explicitly now that I understood only intuitively back then. What I’ve come to understand is that some people feel that offering feedback/criticism is obligatory in certain environments, whether or not feedback is necessary or helpful. That is, if someone were to present a light switch as a simple, effective solution for turning the overhead lights on and off, these sorts of people would suggest a push button switch or The Clapper, not because it was a better solution but simply because they wanted to put in their two cents.

I suspect that there are two main causes of this. The first is that some people very much enjoy expressing opinions, and all matters are subjects for debate. It’s a game of sorts. I have friends that have this attitude, and it can be a great one for driving debates and spurring innovation. It can also be tiresome at times, but it keeps you thinking and on your toes. The other cause I see is less innate and more a manifestation of collaborative environments – people perceive this as obligatory for subtle confirmation of status. Making corrections to the work of ‘inferiors’ reminds everyone present as to their place in the food chain.

Consider a meeting where a more senior peer or a manager reviews work of a less senior peer or subordinate. If the reviewer offers nothing except a “great job”, he may feel as if he’s acknowledging superiority and thus undermining his own role. The feedback thus becomes obligatory, an end rather than a mean, which creates a situation similar in concept to a scheme where police have quotas for traffic tickets given out — it’s assumed that something is wrong, and if nothing is wrong, something must be invented.

This is complicated somewhat by the fact that there is probably something that can be improved about any suggested process, piece of code, idea or anything else, just as it’s safe to assume that there’s always somebody speeding somewhere. As such, there is always a justification for suggestions for improvement or constructive criticism. But, the fact that just about anything could be improved does not necessarily imply that these obligatory suggestions amount to actual improvements. Often, they’re simply time wasters and can even diminish the idea. This is because if the criticism is obligatory, it may not be well thought out or justified – the criticizer may be grasping at straws to assert his authority.

I believe that there are some “tells” as to when this is occurring. If a 10 page document is submitted, and the reviewer offers 2 or 3 criticisms of the first page and then says the rest is fine, this is probably an obligatory criticism. What most likely happened is that the reviewer read until he or she found a few things to express opinions about and then stopped, since the mission of reinforcing authority was accomplished. Another such tell is criticism that misses the point or some obvious fact, for obvious reasons. And then, there is vague or ad hominem criticism — suggestions that the presenter is not up to the task or hasn’t been around long enough to understand things.

To go back to my narrative, I began to see this in action intuitively, pick up on the tells, and manipulate the situation to my advantage. Perfunctory criticism can be ferreted out by inserting a few false flag mistakes early on and allowing them to be corrected so that the idea can be discussed in earnest while placating those seeking reassurance.

So What?

Having read about my own story (and probably either empathizing, as a young developer or remembering back, as a more experienced one), it is worth asking if this is a problem, or if it is simply the way of the world. To that, my answer is “both”. The practice might have some nominal use in keeping people honest and humble, but it has a side effect that outweighs that benefit, in my opinion. It promotes a culture of “tenure over merit” and amplifies the “Dead Sea Effect”, wherein talented new developers tend to leave a company quickly and less enthusiastic and ambitious developers stay around and grandfather into roles of authority.

Alex Papadimoulis also blogged about this effect and said

The reason that skilled employees quit, however, is a bit more complicated. In virtually every job, there is a peak in the overall value (the ratio of productivity to cost) that an employee brings to his company. I call this the Value Apex.

On the first minute of the first day, an employee’s value is effectively zero. As that employee becomes acquainted with his new environment and begins to apply his skills and past experiences, his value quickly grows. This growth continues exponentially while the employee masters the business domain and shares his ideas with coworkers and management.

However, once an employee shares all of his external knowledge, learns all that there is to know about the business, and applies all of his past experiences, the growth stops. That employee, in that particular job, has become all that he can be. He has reached the value apex.

If that employee continues to work in the same job, his value will start to decline. What was once “fresh new ideas that we can’t implement today” become “the same old boring suggestions that we’re never going to do”. Prior solutions to similar problems are greeted with “yeah, we worked on that project, too” or simply dismissed as “that was five years ago, and we’ve all heard the story.” This leads towards a loss of self actualization which ends up chipping away at motivation.

Skilled developers understand this. Crossing the value apex often triggers an innate “probably time for me to move on” feeling and, after a while, leads towards inevitable resentment and an overall dislike of the job. Nothing – not even a team of on-site masseuses – can assuage this loss.

Bruce Webster talks about the role of a frustrating, apparently external, management structure in bringing this phenomenon about and Alex talks about the sense of no longer being able to offer value. Obligatory reviewers short circuit and amplify the Dead Sea Effect by making the frustrating management internal to the group and by setting the “value apex” low right from the start. Talented new hires are more likely to be quickly discouraged with obligatory criticism being directed toward them to reinforce status that reviewers have and they don’t.

What to Do?

I’ve spent a considerable number of words explaining a problem as I see it and an ill-advised ‘solution’ I came up with years back and subsequently discarded, so I’ll offer some thoughts on solutions to this problem. For the shorter tenured, presenter/review-ee worker, there is no entirely self-deterministic solution, but you can improve your odds of success and minimize frustration. The most important thing to do, in my opinion, and the solution that I’ve ultimately adopted is to be extremely well organized and prepared. As you work on your proposal/design/code/etc, document not only what you do, but what other approaches you considered and discarded. If you’re writing actual code, make a list of each class and method and write down its purpose for existence. Organize and format these notes and email them out ahead of the meeting or bring them with you. The effect that this has is to counteract the obligatory objections by showing that you’ve considered and dismissed them as solutions. Obligatory, as opposed to serious or well-reasoned, objections tend to be off the cuff and thus likely to be the same thing that you’ve considered and dismissed, so having an immediate, well-defended answer serves not only to move the review along more quickly to relevant matters, but also to discourage status-seeking reviewers from testing these waters in the future. After all, making obligatory suggestions for modification only works to reinforce status if the attendees of the meeting perceive the suggester as right. Getting burned by a few volleys knocked quickly and decisively back into his court will make the status-seeker lose interest in this as an opportunity for grandstanding.

This practice has other indirect benefits as well. Getting in the habit of always justifying one’s decisions tends to make him better at what he does. If you’re mentally justifying every class and method that you write, I would argue that you’re a lot more likely to have a good design. And, your preparation makes others more effective as well. Thoughtful reviewers will be able to focus during the allotted time on things you hadn’t considered already and make the meeting more productive, and even the status-seekers will at least learn to come more prepared and probe deeper, if only for the purpose of not being shown up (in their minds).

If you’re running a team or a department and have the authority to dictate review policy, I would suggest a few different ideas. First and foremost, I would suggest offering a forum where more junior team members can present without any feedback or criticism at all. This need not be on a production piece of software – but given them some forum to present and be proud of their work without constantly needing to defend it against criticism. People feel pride in their work, for the most part, and getting a chance to showcase that work and feel good about it is important for team morale.

Another strategy would be to have a pool of reviewers and to let the team members choose who conducts the reviews. Keeping track of who they choose provides valuable feedback. I’d wager dollars to donuts that the status-seeker reviewers are rarely, if ever chosen, with developers selecting instead those from whom they can learn and who will make their work better. Of course, they might choose a reviewer who rubber stamps everything, but even that is valuable information to have, as you can talk to that reviewer or else choose a different reviewer.

A third option would be to create some sort of appeals process involving proof of concepts or other forms of ‘proof’ of ideas. If a developer submits a solution using a design pattern, and the reviewer happens not to like that pattern, don’t let that be the end of the discussion. The developer should be given a chance to create a demo or POC to show, concretely, the benefits of using the pattern, or perhaps a chance to cite sources or examples. This serves to create more of a meritocracy than a dictatorship when it comes to ideas.

Is It Worth the Bother?

In the end, one might make a good case that the new developers are wrong frequently enough that stunting their enthusiasm might be good for them, particularly if they’re cocksure about everything. A bite of humble pie is good for everyone from time to time. However, with that line of argument, a fine line emerges between appropriate, periodic humbling and the systematic stifling of new ideas and jockeying for influence. The more that obligatory criticisms and decisions by fiat take hold, the more they are reinforced, to the detriment of the department. I think that it’s of vital important to eliminate contributors to this type of environment to have an efficient team staffed with self-reliant and capable developers, whatever their experience levels may be.

(Clapper image was lifted from this blog post. I chose it as the clapper image source above all the other ones that pop on a google search specifically because I’m very interested in the subject of home automation)

By

Coding Conventions and Ralph Waldo Emerson

Convention or Improvement

I was reading (and commenting on) a blog post by Jimmy Bogard at Los Techies yesterday. The basic premise was that, since Systems Hungarian Notation has basically been rejected by the developer community at large, it is interesting that we continue to pre-pend an “I” to interfaces in C#.

The post and some of the comments touched on the idea that while, yes, this is incongruous with eschewing systems Hungarian, we should continue to do it because it’s what other programmers expect and not following the convention makes your code less intuitive and readable to other C# developers. Jimmy says:

The train for picking different naming styles for interfaces and generic type parameter names left the station a long, long time ago. That ship has sailed. Picking a different naming convention that goes against years and years of prior art…

In the interest of full disclosure, I follow the naming convention and understand the point about readability, but it led me to consider a broader, more common theme that I’ve encountered in my programming career and life in general. At what point do we draw the line between doing something by convention because there is a benefit and mindlessly doing something by convention.

Emerson

Ralph Waldo Emerson famously said, “A foolish consistency is the hobgoblin of little minds.” This is often misquoted as “Consistency is the hobgoblin of little minds,” which, in my opinion, frames my question here well. The accurate quote that includes “foolish” seems to suggest that following convention makes sense, so long as one continually justifies doing so, whereas the latter misses the subtlety and seems to advocate iconoclasm for its own sake, which, one might argue, is a foolish consistency in and of itself. We all know people that mindlessly hate everything that’s popular.

Lest I venture too far into philosophical considerations and away from the technical theme of this blog, I’ll offer this up in terms of the original subject for debate. Is the fact that a lot of programmers do something a valid reason to do it yourself, or is that justification an all purpose cudgel for beating people into following the status quo for its own sake? Is the convention argument a valid one of its own right?

Specifics

This is not the first incarnation in which I’ve seen this argument about convention. I’ve been asked to name method variables “foo” instead of “myFoo” because that’s a convention in a code base, and other people don’t prepend their variables with “my”. Ostensibly, it makes the code more readable if everyone follows the same naming scheme. This is one example in the case of countless examples I can think of where I’ve been asked to conform to a naming scheme, but it’s an interesting one. I have a convention of my own where class fields are pre-pended with underscores, method parameters are camel case, and method variables are pre-pended with “my”.

This results in code that looks as follows:

There is a method to my madness, vis-a-vis readability. When I look at code that I write, I can tell instantly whether a given variable is a class field, method parameter, or method variable. To me, this is clear and readable. What I was asked to do was rename “myOtherSocial” to “otherSocial” thus, in my frame of reference, losing the at-a-glance information as to what is a method parameter and what is a local variable. This promotes overall readability for the many while reducing readability for the few (i.e. me).

That’s an interesting tradeoff. One could easily make the Vulcan argument that better readability needs of the many outweigh the needs of the few. But, one could also make the argument that the many should adapt to my convention, since it provides more information. Taking at face value that I’m right about my way being better (only for argument’s sake – I’m not arguing that there is a ‘right’ in a matter of personal readability and aesthetics) is the idea of convention — that a lot of people do it the other way — a valid argument against improvement?

Knowing When Conventions Can be Ignored

I don’t presume to know what the best naming scheme for local variable or interfaces is. In fact, I’d argue that it’s likely a matter of aesthetics and expectations. People who have a harder time mentally switching contexts are going to be more likely to dig in, and mile-a-minute, flighty thinkers will be more likely to get annoyed at such ‘pettiness’ and ‘foolish consistency’. Who’s right? Who knows – who cares.

Where the rubber meets the road is how consistency or lack thereof affects a development team as a whole. And, therein lies the subtlety of what to do, in my mind. If I am tasked with developing some library with a small, public facing API for my part in a team, it makes the most sense for me to write code in a style that makes me most productive. The fact that my code might be less readable to others will only be relevant at code reviews and in the API itself. The former is a one and done concern and the latter can be addressed by establishing “public” conventions to which I would then adhere. Worrying about some hypothetical maintenance programmer when deciding what to name and how to case my field variables is fruitless to a degree because that hypothetical person’s readability preferences are strictly unknowable.

On the other hand, if there is a much more open, collaborative paradigm where people are constantly modifying one another’s code, then establishing conventions probably makes a lot of sense, and following the conventions for their own sake would as well. The conventions might even suck, but they’ll be consistent, and probably save time. That’s because the larger issue with the consistency/convention argument is facilitating efficiency in communication — not finding The One Try Way To Do Things.

So, the “arbitrary convention” argument picks up a lot of steam as collaboration becomes more frequent and granular. I think in these situations, the “foolish consistency” becomes an asset to practical developers rather than a “hobgoblin of little minds.”

Be an Adapter Instead of an Evangelizer

All that said, the conventions and arguments still annoy me on some level. I can recognize the holistic wisdom in the approach while still being aggravated by being asked to do something because a bunch of other people are also doing it. It evokes images in my head of being forced to read “Us Weekly” because a lot of other people care how many times Matt Damon went to the gym this week. Or, more on subject, it evokes images of being asked to prepend “str” to all my string variables because other people think that (redundant in Visual Studio) information makes things more readable.

But, sometimes these things are unavoidable. One can go with it, or one can figure out a way to make all parties happy. I blogged previously about my solution to my own brushes with others’ conventions. My solution was a DXCore plugin that translated my conventions to a group’s, and vice-versa. I’m happy, the group is happy, and I learned stuff in the process.

So, I think the most important thing is to be an adapter. Read lots of code, and less will throw you off your game and cause you to clamor for conventions. Figure out ways to satisfy all parties, even if it means work-arounds or interesting solutions. Those give your brain some exercise anyway. Be open to both the convention argument and the argument for a new way of doing things. Go with the flow. The evangelizers will always be there, red faced, arguing that camel case is stupid and only pascal case makes sense. That’s inevitable. It’s not inevitable that you be one them.

(Image compliments of: http://geopolicraticus.wordpress.com/2009/01/28/short-term-thinking/ )

By

More and More With TDD

TDD Revisited Again

Previously I blogged about TDD with some reservations and then later as I re-introduced myself to the pure practice as I had learned it some many moons ago. For the last month, I have been plowing ahead methodically with this experiment, and I thought I’d revisit some of my thoughts from earlier.

Reinforced Pros

First off, the pros still stand in my mind, but I’d like to add one enormous additional pro, which is that you get code right the first time. I mean, really, really right. Right as in coding for a few days without ever running the application you’re working on, and then firing it up and having it work flawlessly the first time you run it.

And, I’m not talking about plodding along with some tiny development exercise like scoring a bowling game. I mean, I recently wrote an application that would parse a text file, format and filter records, and insert the result in a database. Using a combination of Moq for my internal seams and Moles for external concerns (file and database), I exhaustively tested everything I wanted to do as I went, following the red-green-refactor process to the letter. After some hours here and there of working on this (shaking off my TDD rust being part of the time spent), I finally hit F5 and it worked. My files were parsed, my databases updated, my log file written accurately. This wasn’t a fluke, either, because I repeated the process when I wrote another utility that piggy-backed on this first one to send out emails with the results. I wrote an API for setting up custom mail messages, dependency injection for different mail clients, and wireup code for integrating. Never once did I step through the debugger trying to figure out what network credential I had missed or which message was missing which formatting. I put the finishing touches on it with my tests passing and code coverage at 100%, hit F5 and had a well-formed, perfect message in my inbox before I even looked away from my screen, expecting to be kicked into the debugger. That is a remarkable pro.

I mean, think of that. If you’re a skeptic about TDD or unit testing in general, you’ve probably never experienced anything like that. If you have, that’s amazing. I don’t know anyone that doesn’t develop in this fashion that has this happen. I mean, people who are really sharp get it mostly right, and sometimes just have to tweak some things here and maybe step through a few functions a few times, but without ever running the code? I’m not buying it. And even if they did, I come out way ahead this way. A non-tested code base is going to be harder to modify. I have hundreds of passing tests now that will tell me if subsequent changes break anything.

Cons Mitigated

So, I’d like to take a look back at the cons, and revisit my opinions after a month of doggedly sticking with TDD:

  1. Larger design concepts do not seem to be emergent the way they might be in a more shoot from the hip approach — figuring out when you need a broader pattern seems to require stepping out of the TDD frame of mind
  2. There is no specific element of the process that naturally pushes me away from the happy path — you wind up creating tests for exceptional/edge cases the way you would without TDD (I guess that’s not strictly a con, just a lack of pro).
  3. It becomes easier to keep plodding away at an ultimately unworkable design – it’s already hard to throw work you’ve put in away, and now you’re generating twice as much code.
  4. Correcting tests that have been altered by changing requirements is magnified since your first code often turns out not to be correct — I find myself refactoring tests as often as code in the early going.
  5. Has the potential to slow development when you’re new to the process — I suppose one might look at this as a time investment to be paid off when you get more fluid and used to doing it.

Regarding (1) and (3), this seems not to be the issue that I thought it would be. I had grown used to a fast flurry trying out a class concept, and realizing what refactorings needed to be done on the fly, causing it to morph very quickly. I had gotten very proficient at ongoing refactoring without the red-green preceding it, so, in this way, I was able to ‘fake’ the TDD benefit. However, I realized a paradoxical benefit – TDD forced me to slow down at the beginning. But rather than really slowing me down on the whole, it just made me toss out fewer designs. I still refactor with it, but it isn’t necessary as often. The TDD forces me to say “what do I want out of this method/class” rather than to say “what should this do and how should it relate to others”. The difference is subtle, but interesting. The TDD way, I’m looking at each method as an atom and each class as a unit, whereas the other way I’m putting the cart a bit before the horse and looking at larger interactions. I thought this was a benefit, but it resulted in more scratch out work during prototyping than was really necessary. I was concerned that TDD would force me to throw away more code because I’d be tossing tests as well as code, but in reality, I just toss less code.

Regarding (2), this has sorted itself out as well. TDD does tend to lead you down the happy path as you’re using it to incrementally correct and refine a method’s behavior, but there’s nothing that stops you from tossing in along with that a thought of “how do I want this to handle an exception in a method that it calls?” This, I’m finding is a discipline and one that I luckily brought with me from testing after or testing during. It is not incompatible with TDD, though TDD does help me not get carried away focusing on corner cases (which can be ferreted out with tools like Pex or smoke tests later).

Regarding (4), this has also proved not to be too much of an issue, though I have seen a touch of this. But, the issue really comes in when it turns out that I don’t need some class, rather than a swath of tests. TDD helps my classes be better thought out and more decoupled, and so, I’ve found that when I’m cutting things, it tends to be surgical rather than a bloodbath.

Regarding (5), this is true, but I’ve fought through the burn. However, there is another thing that goes on here that I’d point out to people considering a foray into TDD. Imagine the scenario that I described above, and that you’re the non-TDD hare and I’m the TDD tortoise. I’m plodding along, red-green-refactoring my parser when you’ve finished yours. You’re probably running it on an actual file and making changes while I’m still working methodically. By the time I’m finished with my parser and doing the formatting, you’ve got your formatting finished and are working on the database. By the time I get to the database, you’ve already got things working 90%, and are busy stepping through the debugger and cursing the OleDbCommand API.

But, that’s where things start to get interesting. Often times, in my experience, this is the part of development that has you at the office late, pulling your hair out. The application is 90% done at 4:00, and you’re happy because you just need to make a minor tweak. Suddenly, it’s 8:00 and you’re hungry and cranky and cursing because no one is left in the office. That’s a brutal 4 hours specifically because you imagined in to be 20 minutes at 4:00. But, let’s say you finally get it and go home. I’ll finish an hour or so after you and you win the first leg of the race, though I’ll do it without a lot of fanfare and frustration, since I’m never hitting snags like that with my methodical process.

But now, an interesting thing happens. We send our versions out for Beta testing, and the issues start coming back. You have your own knowledge of your code to go on as you fix defects and perform minor refactorings, while I have 100% code coverage. I quickly find issues, fix them, see all green and am ready to deliver. You quickly find issues, fix them, and then run the application a bunch of times in different scenarios trying to ferret out anything that you could possibly have broken. I’m gaining on you fast. And, I’m going to keep gaining on you as the software grows. If Beta lasts more than a short time, I’ll win. If there are multiple versions of the software, I’ll win easily. At some point, I’ll lap you as you spend way more time repairing defects you’re introducing with each change than fixing the one you set out to fix. The time investment isn’t new TDD versus seasoned TDD. It’s a little time up front for a lot of time later.

Eating Crow

I had tried TDD in the past, but I think I wasn’t ready or doing it right. Now, I’m convinced that it’s definitely the path for me. Somewhere, I saw a presentation recently that mentioned sort of an arc of unit testing. People tend to start out with test-last, and then move to test during, then test-first and then test-driven, according to this presentation (I wish I remember whose it was so I could link and give credit). I think my previous issue was that I had short circuited the arc and was trying to assimilate too much all at once. If you’re a developer trying to convince others of the merits of unit tests, TDD might be a lot to spring on them up front.

I was thinking about why that might be, and something occurred to me from way back in my undergrad days. The first thing we all tend to do is want to write all of the code at once. Since we’re given small assignments to start, this works. It’s hard to turn bubble sort into a lot of different feedback passes — you write the whole thing and then debug the ‘finished’ product. As classes get harder, you try to keep this dream alive, but at some point you have the “aha” moment that you need to get shorter feedback cycles to prevent assembling the whole thing and getting it all wrong. So, you do this by making your code modular and developing components. That carries you through into the early part of your career, and you have a natural tendency to want to get code up and running as quickly as possible for feedback. Then, TDD comes along and changes the nature of the feedback altogether. You’re no longer getting the feedback from your running application, but from your unit tests of small components. So, if you’re building a house app, you don’t start with the House class and run it. Instead, you start with a “Bedroom” class, or even a “Bed” class. While your counterparts are adding rooms to their running houses, you’re getting your “Bed” class working flawlessly. And, while they’re cobbling furniture into their rooms, you’re assembling rooms from their furniture. When they’re debugging, you’re finally ready for your first “F5” feedback. But, you haven’t come full circle and turned into that freshman with bubble sort – you’ve gotten real feedback every 30 seconds or so. You’ve gotten so much feedback, that there’s little anticipation with the first real run of the component you’re working on for that week. You’re pretty sure it’s going to work and you’re going to be heading home at 5:00.

So, yeah, I guess I’m a TDD convert, and will eat some crow for my earlier posts.

By

Using Moles with the System Assembly

Short but sweet post tonight, and I’m mostly putting this here for reference. It’s annoyed me a few times, and each time I have to google around until I find something like this stack overflow post.

If you add a moles assembly for System, unlike other framework assemblies, building your test project will blow up spectacularly with too many compiler errors to count. I don’t know all the whys and wherefores, but I do know that it’s easily remedied as hveiras says. You simply change:

to

in your System.Moles file that’s generated. That’s it.

Philosophically, it seems like a bit of an oversight that this would be necessary, as this never fails to fail out of the box. There may be something I’m missing, but it seems like it’d be nice if this attribute were added by default for the System Assembly. In any case, I (and you) have it here for posterity now, though probably the act of explicitly typing it out has rendered it moot for me, as documenting things usually makes me remember them and not need my documentation… which reminds me – I’m probably about due for a periodic re-reading of one of my all time favorite books.