DaedTech

Stories about Software

By

TDD Chess Game Part 4: Getting Organized

Alright, welcome back to this series.

A couple of housekeeping things:

  1. I have bitten the bullet and used the Visual Studio White theme along with 14 point font to record, so hopefully the videos going forward should be easier to watch. It’s a little surreal to work with, but c’est la vie.
  2. The source code is now available on github for you to follow along. The coding is usually running ahead of my publication, so if you want to see the code from a given video, you may have to grab a slightly earlier version.

Here’s what I accomplish in this clip:

  • Started using a little todo list to keep track of what I’ve done and what I need to do.
  • Cleaned up code as reported by static analysis tools.
  • Pulled some production classes into their own namespaces and out of the test classes.
  • Defined an abstract Piece class.
  • Defined a second inheritor, “Rook,” for Piece.
  • Defined a bit of dumb functionality for Rook’s “GetMovesFrom” to get it started.
  • Implemented ability for a pawn to move two spaces on its first move.
  • Defined a piece concept of “HasMoved.” (albeit just for Pawn)

And here are the lessons to take away:

  • Keeping a list of smallish things you want to change can help you keep track of what needs to be done without distracting you too much (I picked this technique up from Kent Beck’s “Test Driven Development By Example.”)
  • If you’re using NCrunch, use the green dots being dark or bright as a quick way to tell if the code is compiling.
  • Gamify cosmetic issues. If “Optimize Namespaces” and things like that are important, make violations ugly and distracting in the IDE and you’ll get annoyed and fix them whereas you probably wouldn’t bother, otherwise.
  • It’s okay to write stupid tests if you do so knowing that you’ll fix them. Finding ways to always write a test to change production code is good for practicing the TDD discipline until it starts to become second nature.
  • It’s okay to write a test that causes a non-compile failure and then needing to do a good bit of work to get everything back to compiling/passing.
  • I’ve mentioned this previously, but it bears repeating: it’s okay to reuse a test (especially a stupid one) to get a failing test.
  • If you weren’t aware of C# yield keyword and deferred execution, it’d be a good thing to familiarize yourself with.
  • Force yourself not to copy and paste as much as possible, even when it seems dumb. Feeling the pain of re-typing things will make it painfully obvious when you’re duplicating code and could do something better.

And, here’s the clip:

By

Implied Acceptance Criteria

I’m on a Scrum team these days, serving as Product Owner, and I was watching developers do functionality demos. This has generally been of the form of them walking me through the happy path, and then me taking it for a test drive and trying to put myself in a user, ‘cleverly’ typing “twenty five” into a text box that clearly wants an integer to see what happens. Yellow screen of death? Generic error message? Scornful validation message such as “dude, what’s wrong with you?” Helpful validation message such as “your response for how many children you have cannot be a negative number, a decimal, or typeset nonsense?”

If what happens isn’t what I think should happen, it’s then off to check out the acceptance criteria/tests. Did it say anything in there about a validation message? Should it have to? Are there things that ought to be “universal acceptance criteria?”

To me, the answer to that question is, “sort of, but I’d prefer to think of them more as default/implied ACs.” And so I started to enumerate them on an internal wiki, kind of as an exercise for myself to see if there were certain things that I think should always be true. Maybe these are like the equivalent of code contracts, but contracts between the developer and the user. Here are some ideas that I came up with:

  • When user supplies invalid input, a message should be shown explaining why the input was not accepted.
  • The way to get back to where you just were should always be available and obvious (e.g. “cancel” button or “back” link or something).
  • Nothing that happens should ever result in a “yellow screen” or whatever equivalent indicates to the user that whatever is going wrong is something you never dreamed of (for production release, anyway — I have no issues with crashes like this in internal or beta tests as part of a “fail early” strategy)
  • If an exception occurs that the user would understand, it is explained to the user (e.g. “The connection to the database was lost.”)
  • There is never a point during the normal course of usage where the user thinks, “should I just, like, wait, or is it frozen?”
  • If it’s possible to prevent the user from doing the wrong thing, the user is prevented from doing the wrong thing (e.g. if user clicks “save” and that’s a long running operation, “save” button is disabled until it’s okay to click again)

These are some things that I think ought to be true of your application’s behavior unless there is a good reason to deviate. These are things that I generally think of when I’m working, even if nothing is spelled out explicitly. I think of these and I think of more, depending on the application context (e.g. is this something that should be permission controlled or is this something where the verbiage might be changed later?) Over the years, I’ve developed a pretty lengthy mental checklist for what I consider good application experience, even without consciously realizing that I’ve done so for the most part. But I think it’s important for us to try to grow this evoked set in our minds as we go.

So what about you? Do you have anything you’d add to this list — any glaring omissions? I’m actually looking to tabulate some of these things into a working document and perhaps expand it to cover other kinds of minimal checklists for various scenarios (such as testing your own code via running the application, finding and fixing a bug, etc). Perhaps when I get it more fleshed out at some point, I’ll revisit and post again, but either way, I’d be interested to hear what you consider to be table stakes/minimum standards for how your applications interact with users.

By

Getting Started on the Roslyn Journey

It’s not as though it’s new; Roslyn CTP was announced in the fall of 2011, and people have been able to play with it since then. Roslyn is a quietly ground-breaking concept — a set of compilers that exposes compiling, code modeling, refactoring, and analysis APIs. Oh, and it was recently announced that the tool would be open source meaning that all of you Monday morning quarterback language authors out there can take a crack at implementing multiple inheritance or whatever other language horrors you have in mind.

I have to say that I, personally, have little interest in modifying any of the language compilers (unless I went to work on a language team, which would actually be a blast, I think), but I’m very interested in the project itself. This strikes me as such an incredible, ground-breaking concept and I think a lot of people are just kind of looking at this as a curiosity for real language nerds and Microsoft fanboys. The essential value in this offering, to me, is the standardizing of code as data. I’ve written about this once before, and I think that gets lost in the shuffle when there’s talk about emitting IL at runtime and infinite loops of code generation and whatnot. Forget the idea of dispatching a service call to turn blobs of text into executables at runtime and let’s agree later to talk instead about the transformative notion of regarding source code as entity collections rather than instruction sheets, scripts, or recipes.

But first, let’s get going with Roslyn. I’m going to assume you’ve never heard of this before and I’m going to take you from that state of affairs to doing something interesting with it in this post. In subsequent/later posts, we’ll dive back into what I’m driving at philosophically in the intro to this post about code as data.

Getting Started

(Note — I have VS2013 on all my machines and that is what I’ve used. I don’t know whether any/all of this would work in Studio 2012 or earlier, so buyer beware)

First things first. In order to use the latest Roslyn bits, you actually need a fairly recent version of Nuget. This caught me off guard, so hopefully I’ll save you some research and digging. Go to “Tools” menu and choose “Extensions and Updates.” Click on the “Updates” section at the left, and then click on “Visual Studio Gallery.”

NugetUpgrade

If you’re like me, your version was 2.7.something and it needs to be 2.8.1something or higher. This update will get you where you need to be. Once you’ve done that, you can simply install the API libraries via Nuget command line.

With that done, you’re ready to download the necessary installation files from Microsoft. Go to http://aka.ms/roslyn to get started. If you’re not signed in, you’ll be prompted to sign in with your Microsoft ID (you’ll need to create one if you don’t have one) and then fill out a survey. If you get lost along the way, your ultimate destination is to wind up here.

At this point, if you follow the beaten path and click the “Download” button, you’ll get something called download.dlm that, if your environment is like mine, is completely useless. So don’t do that. Click the circled “download” link indicated below to get the actual Roslyn SDK.

DownloadRoslyn

Once that downloads, unpack the zip file and run “Roslyn End User Preview” to install Roslyn language features. Now you can access the APIs and try out interesting new language features, like this one:

That’s all well and good for dog-fooding IDE changes and previewing new language features, but if you want access to the coolness from an API perspective, it’s time to fire up Nuget. Open up a project, and then the Nuget command line and type “Install-Package Microsoft.CodeAnalysis -Pre”

Once that finishes up, make your main entry point consist of the following code:

At this point, if you hit F5, what you’re going to see on the screen is a list of the fields contained in the class that you specify as your “sourceCodePath” variable (at least you will with the happy path — I haven’t tested this extensively to see if I can write classes that break it). Now, could you simply write a text parser (or, God forbid, some kind of horrible regex) to do this? Sure. Are there C# language modeling utilities like a Code DOM that would let you do this? Sure. Are any of these things the C# compiler? Nope. Just this.

So think about what this means. You’re not writing a utility that uses a popular C# source code modeling abstraction; you’re writing a utility that says, “hey, compiler, what are the fields in this source code?” And that’s pretty awesome.

My purpose here was to give you a path from “what’s this Roslyn thing anyway” to “wow, look at that, I can write a query against my own code.” Hopefully you’ve gotten that out of this, and hopefully you’ll go forth, tinker, and then you can come back and show me some cool tricks.

By

NCrunch and Continuous Testing: The Must-Have Setup

Most of this post was taken from the transcript of my Pluralsight course on NCrunch. If you are interested in watching the course but are not a Pluralsight subscriber, feel free to email me or leave a comment requesting a trial, and I’ll get you a 7 day subscription to check it out.

Understanding the Legitimate, Root-Cause Objection to TDD

In my experience, there are three basic “camps” of reactions to the concept of test driven development (TDD) from those not experienced with it: willing students, healthy skeptics and reactionary curmudgeons. The first group is basically looking for a chance to practice and needs no convincing. The last group will have to be dragged along, kicking and screaming, and so there’s no persuading them without the threat of negative consequences. It is the middle group that tends to have rational objections, some of which are well-founded and others of which aren’t so much. A lot of the negative reaction from this group is the result of reacting to the misconceptions that I mentioned in this post about what TDD is and isn’t. But even once they understand how it works, there are still some fairly common and legitimate objections that are not simply straw man arguments.

  1. The most common and prevalent objection is that coding this way means that you’re doing a lot more work. You’re taking more time and writing more code and people don’t necessarily see the benefit, especially in cases where they already know what code they want to write.
  2. Many of the misconception objections and other inexplicable resistance is really the result of people simply not knowing how to write tests or practice TDD, and perhaps at times being reluctant to admit it. Others may freely admit it. Either way, the objection is that TDD, like any other discipline, would take time to learn and require an investment of effort.
  3. There is also more code that is going into a project since you now have an additional test class for each single class you would otherwise have created. More code means more maintenance time and effort.
  4. Many astute observers also realize that a lot of legacy code, particularly that involving large-work constructors, singletons, and static state is very hard to test, making attempts to do so effort-intensive.
  5. And, along the same lines , they also realize that there would be more effort required than simply learning how to do TDD – it would also mean learning different design techniques such as dependency injection, polymorphism, and inversion of control.

When you consider all of these objections, they all have a common thread. At the core of it, they’re really all variants on the theme of not having enough time. Writing the tests, maintaining the test code, learning new ways of doing things, and applying them to new and old code are all things that take time, and for most developers, time is precious. Someone selling TDD is a lot like someone selling you on a 401K: they’re convincing you that sacrificing now is going to be worth it later and asking you to take this, to some degree, on faith.

Could TDD be better?

Justifying the adoption of TDD to a healthy skeptic hinges largely on demonstrating that it provides a net benefit in terms of time, and thus cost. So how can these objections be reconciled and the concerns addressed?

Well first up are the learning curve oriented objections. And the truth is that there’s no way around this one being a time sink. Learning how to do TDD and learning how to write testable code are going to take time, no matter what. If you do not have the time to learn, this is a perfectly valid objection, but only in the shorter term. After all, we work in an industry where change is the only constant and learning new languages, frameworks, and methodologies is pretty much table stakes for staying relevant.

Regarding development time overall, a very common argument made by TDD proponents is that the practice saves time over the long haul. This is reminiscent of the parable of the tortoise and the hare where the TDD practitioner is a tortoise plodding along, getting everything right and the hare is generating reams of code quickly but with mistakes. The hare will declare himself done more quickly, but he’ll spend a lot more time later troubleshooting, reading log files, debugging, and fixing errors. The tortoise may not finish as quickly, but when he does, he truly is done.

But what about in the short term? Is there anything that can be done to make things go more quickly in the short term for TDD practitioners? Could we strap a rocket pack to the tortoise and make him go faster than the hare while preserving his accuracy?

RocketTurtle

Speeding up the Feedback Loop

What if I told you a story? What if I told you that you could write code and know whether or not it was working nearly instantaneously? In this world of development, you don’t have to wait while your application starts up, and then navigate through various user interface screens to get to the action that will trigger the bit of code you want to verify. There is no more repetitive clicking and typing and waiting for screens to load. In fact, in this world you don’t even need to build your project or compile your code. All you need to do is type and see, as you’re typing, whether or not the changes you’re making are right. And, you can see a visual metric for how much confidence you can have in your changes by virtue of how much your code is covered by the unit tests.
Does that story sound too good to be true? Well, I’ll admit that it does sound pretty good, but I’ll let you in on a little secret – it is true. There is a name for this paradigm, and it’s called “continuous testing.” And there are various tools out there for different platforms that make it a reality, right now as we speak.

To understand the magic of continuous testing, it’s essential to understand one of the most important, but often overlooked, concepts in computer science. I’m talking about the feedback loop. At its core, programming is a series of experiments. Whenever you approach a programming task, you have a code base that does something, and you have a goal to make it do something different or new. To achieve this goal, you identify intermediate behaviors that you’d like to see to mark progress, and then you make changes that you think will result in those behaviors. Then, you run the application to see if what you thought would happen does, in fact, happen.

For example, perhaps you want to have your application display customer information stored in a database to the screen when the user clicks a certain button. You might first say “forget the database – let’s just get the button click to result in some hard-coded value being displayed,” and then set about altering the code to make that happen. When you’d made your changes to the code, you’d run the program and click that button to see what had happened.

Considered closely, this process is actually a lot like the scientific method. For step (1) you read the code. For step (2) you hypothesize what you’ll need to do to the code. For step (3) you predict the outcome of your changes, and for step (4) you make the changes and observe the results. The amount of time that it takes to perform an iteration of your coding version of the scientific method is what I’m calling the “feedback loop.” How long does it take for you to have an idea, implement it, and verify that it had the desired effect?

Scientist

In the early days of programming when the use of punch cards was common, feedback times were very lengthy. Programmers would reason carefully about everything that they did because feedback times were extremely slow, meaning mistakes were very costly. While many improvements have been made across the board to feedback times, situations persist to this day when the feedback loop is excruciatingly slow. This includes long running or resource-intensive applications and distributed systems with high latency. With such systems, programmers on projects often devise schemes to try to shorten the feedback loop, such as mocking out bottlenecks to allow fast verification of the rest of the system.

What they’re really trying to do is shorten the feedback loop to allow themselves to be more productive. When a great deal of time elapses between trying something and seeing what happens, attention tends to wander to distractions like twitter or reddit, exacerbating the inefficiency in this already-slow process. Developers innately understand this problem and are frustrated by the long build and run times of behemoth and slow-running applications.

To combat this problem, developers intuitively favor faster schemes. Ask yourself whether you prefer to work on a small project that builds quickly or a large one. How about a slow test suite versus a fast one? By speeding up the feedback loop you trade frustration and wandering attention span for engagement and a feeling of accomplishment. Techniques like relying on fast-running unit tests and keeping modules small and decoupled help a great deal with this, but we can get even faster.

If short feedback is good, immediate is definitely better. Anyone who has done extensive work at a command line or, in general used a Read-Evaluate-Print-Loop (REPL) understands this. Attention does not wander at all during a session like this. Historically, such a thing wasn’t possible in a compiled language, but with the advent of multicore systems and increasingly sophisticated compiler technology, times are changing. It is now possible to have a build running in the background of an IDE like Visual Studio even as you modify the code.

NCrunch

If you’ve been watching my series on building a chess game using TDD you couldn’t help but notice the red and green dots on the left side of the code window, since they catch the eye. What you were seeing was the tool, NCrunch, in action. Now it’s time to get properly acquainted.

NCrunch is a software product by Remco Software and was written by software developer Remco Mulder, who owns the company. It is a tool written specifically to allow developers to practice continuous testing in Visual Studio. NCrunch is a commercial product with a tiered pricing model and full-blown customer support. And it operates as a plugin to Visual Studio so there is no need to integrate or operate any kind of standalone application. It drops right in, comfortably with a tool with which you are already familiar.

For the first several years of its existence, NCrunch was free, since it was in an extended state of Beta release. During the course of these years, it grew a substantial and loyal user base. In the fall of 2012, Remco decided to issue version 1.0 and release NCrunch as a full, commercial product with a licensing model and production support. It is now on version 2.5 and is most certainly an excellent, commercial-grade product that is worth every penny.

As I write my code using this tool, you may notice things that I rarely or never do. I rarely, if ever, run an application. I rarely, if ever, use the unit test runner. I rarely even compile my code, though I do this sometimes simply because I happen to be quite accustomed to looking at compiler feedback in the errors window. Continuous testing tools like NCrunch may have been a novelty when they came out, but I would argue that they’re rapidly becoming table stakes for efficient development these days.

Before NCrunch, the viability of TDD for me was tied up in the idea that investing extra time up front meant that I wouldn’t later be revisiting my code, debugging, tweaking, fixing, when I was further removed and it’s more time consuming. With NCrunch, I don’t even need to make that case. Now, if you took TDD and NCrunch away, my development process would be substantially slower as I sat there waiting for the application to compile or the test runner to do its thing.

If you don’t have this, get it. You won’t be sorry. Forget clean code, unit testing, TDD, all of that stuff (well, not really — but indulge me here for a second). Just get this setup for the tight feedback loop alone. There is nothing like the feeling of productivity you get from typing a line of code and knowing in less than a second, without doing anything else, whether the change is what you want. That incredible power makes it all worth it — the learning curve of the tool, the cost of the tool, adopting TDD, learning to unit test. It’s like getting a car with 500 horse power and feeling that acceleration; it ruins you for anything less.

By

TDD Chess Game Part 3: Stumbling and Refactoring

My apologies. I meant to be a little more regular in this series, but I stumbled a bit out of the gate, as I got into the home stretch of my next Pluralsight course. Now that the course is delivered (not released yet – in the review/edit phase), I have some more time, so I’m planning to pick this back up and go with it a little more regularly.

One interesting thing that arises out of these “fits and starts” kind of passes at it is that it mimics an actual, common development scenario: spotty maintenance coding. What I mean is, so many TDD series that you’ll watch or coding dojo/exercises in which you’ll participate have a premise that you have some fixed length of time during which to pay complete attention. But in this series, I’m sort of poking at it for 10 minutes here and 20 minutes there, very seriously mimicking an environment where you’re plugging a lot of holes, thrashing a bit and saying, “where was I and what was I dong here?”

That’s evident in this clip, probably a little too much for me really to call it polished. And as such, I don’t accomplish a ton, but here’s what I did accomplish (not necessarily in order):

  • Tied up a loose end by getting rid of the last of the primitive obsession passing of x and y coordinate ints.
  • Implemented sanity precondition checks for the input Board’s AddPiece() method, in terms of where pieces could be placed.
  • Pushed functionality for validating coordinates into the coordinate itself.
  • Eliminated duplication in the validation with a refactoring.

And, here are some lessons to take away from this, both instructional from me and by watching me make mistakes:

  • After a conceptual refactoring, such as replacing multiple primitives with a type, take a look around to make sure you cleaned up all instances of the former.
  • When you’re not really sure what to do next (i.e. “coder’s block” or “paralysis by analysis”), implement some sanity checks for preconditions/invariants. This might jolt you into some next steps as you do it.
  • Make ABSOLUTELY SURE that a test goes red when you think it should go red. Not understanding why a unit test is passing is just as bad as not understanding why it’s failing. In both cases, it means you don’t understand what your code is doing. Stop everything and get your brain in sync with the code immediately to save yourself a lot of frustration later. (See “programming by coincidence,” which I saw coined in the book “The Pragmatic Programmer” — and then, don’t do it!)
  • You’re going to make mistakes. Often dumb ones. The beauty of TDD and its fast feedback loop is to prevent them from festering and being worse later.
  • This is more of an editorial/opinion take, but I’ve more recently gravitated toward allowing my TDD to include what might be called “integration tests” (tests that exercise the interaction between two classes). As long as the test makes sense from a behavioral standpoint and provides clarity, I think it’s fine. Some, particularly those in the BDD camp, even argue that this is preferred, and that your tests should really only go through the outer API of your module/application.
  • Eliminate duplication, however trivial and however subtle. If you see repetition of any kind, you can probably extract a method. Some productivity tools and IDEs will even help you locate possible duplication.

Finally, a few notes on the video itself (and resultant code):

  • For those of you who suggested a larger font size, look for that in part 4. I apologize, but I had actually recorded the video for this already when I was taking suggestions. In the production, I did zoom to a slightly smaller area, so we’ll see if that helps any.
  • I had one commenter express a preference for a white background instead of the VS Dark theme that I use. White work-spaces give me a headache, so I darken all IDEs and things that I work in. For one person, I don’t think I’ll pull the trigger, but if more people start responding and expressing that preference, I’ll agree to suck it up and change colors.
  • The code is now on github. I’ll commit the code each time I record the video and tag it with a comment corresponding to the part of the series in question. The initial push to master just reads “Initial publish to Github” but it corresponds to the code at the end of this clip. From here forward, I’ll sync them, though if you check the repo, it’ll probably run slightly ahead of me publishing the videos because I record the audio and do these writeups after the fact.
  • Again, the higher res you view this in the better. I’d go for 1440P if you can.

By

What To Return: IEnumerable or IList?

I’ve received a couple of requests in various media to talk about this subject, with the general theme being “I want to return a bunch of things, so what type of bunch should I use?” I’m using the term “bunch” in sort of a folksy, tongue-in-cheek way, but also for a reason relating to precision — I can’t call it a list, collection or group without evoking specific connotations of what I’d be returning in the C# world (as those things are all type names or closely describe typenames). So, I’m using “bunch” to indicate that you want to return a “possibly-more-than-one.”

I suspect that the impetus for this question arises from something like a curt code review or offhand comment from some developer along the lines of “you should never return a list when you could return an IEnumerable.” The advice lacks nuance for whatever reason and, really, life is full of nuance. So when and where should you use what? Well, the stock consultant answer of “it depends” makes a good bit of sense. You’ll also probably get all kinds of different advice from different people, but I’ll describe how I decide and explain my reasoning.

First Of All, What Are These Things?

Before we go any further, it probably makes sense to describe quickly what each of these possible return values is. IList is probably simpler to describe. It’s a collection (I can use this because it inherits from ICollection) of objects that can be accessed via indexers, iterated over and (usually) rearranged. Some implementations of IList are readonly, others are fixed size, and others are variable size. The most common implementation, List, is basically a dynamic array for the sake of quick, easy understanding.

I’ve blogged about IEnumerable in the past and talked about how this is really a unique concept. Tl;dr version is that IEnumerable is not actually a collection at all (and it does not inherit from ICollection), but rather a combination of an algorithm and a promise. If I return an IEnumerable to you, what I’m really saying is “here’s something that when you ask it for the next element, it will figure out how to get it and then give you the element until you stop asking or there are none left.” In a lot of cases, something with return type IEnumerable will just be a list under the hood, in which case the “strategy” is just to give you the next thing in the list. But in some cases, the IEnumerable will be some kind of lazy loading scheme where each iteration calls a web service, hits a database, or for some reason invokes a 45 second Thread.Sleep. IList is (probably) a data structure; IEnumerable is a algorithm.

Since they’re different, there are cases when one or the other clearly makes sense.

When You’d Clearly Use IEnumerable

Given what I’ve said, IEnumerable (or perhaps IQueryable) is going to be your choice when you want deferred execution (you could theoretically implement IList in a way that provided deferred execution, but in my experience, this would violate the “principle of least surprise” for people working with your code and would be ill-suited since you have to implement the “Count” property). If you’re using Entity Framework or some other database loading scheme, and you want to leave it up the code calling yours when the query gets executed, return IEnumerable. In this fashion, when a client calls the method you’re writing, you can return IEnumerable, build them a query (say with Linq), and say “here, you can have this immediately with incredible performance, and it’s up to you when you actually want to execute this thing and start hammering away at the database with retrieval tasks that may take milliseconds or seconds.”

Another time that you would clearly want IEnumerable is when you want to tell clients of your method, “hey, this is not a data structure you can modify — you can only peek at what’s there. If you want your own thing to modify, make your own by slapping what we give you in a list.” To be less colloquial, you can return IEnumerable when you want to make it clear to consumers of your method that they cannot modify the original source of information. It’s important to understand that if you’re going to advertise this, you should probably exercise care in how the thing you’re returning will behave. What I mean is, don’t return IEnumerable and then give your clients something where they can modify the internal aggregation of the data (meaning, if you return IEnumerable don’t let them reorder their copy of it and have that action also reorder it in the place you’re storing it).

When you’d clearly use IList

By contrast, there are times when IList makes sense, and those are probably easier to understand. If, for instance, your clients want a concrete, tangible, and (generally) modifiable list of items, IList makes sense. If you want to return something with an ordering that matters and give them the ability to change that ordering, then give them a list. If they want to be able to walk the items from front to back and back to front, give them a list. If they want to be able to look up items by their position, give them a list. If they want to be able to add or remove items, give them a list. Any random accesses and you want to provide a list. Clearly, it’s a data structure you can wrap your head around easily — certainly more so than IEnumerable.

Good Polymorphic Practice

With the low hanging fruit out of the way, let’s dive into grayer areas. A rule of thumb that has served me well in OOP is “accept as generic as possible, return as specific as possible.” This is being as cooperative with client code as possible. Imagine if I write a method called “ScareBurglar()” that takes an Animal as argument and invokes the Animal’s “MakeNoise()” method. Now, imagine that instead of taking Animal as the parameter, ScareBurglar took Dog and invoked Dog.MakeNoise(). That works, I suppose, but what if I had a guard-bear? I think the bear could make some pretty scary noises, but I’ve pigeon-holed my clients by being too specific in what I accept. If MakeNoise() is a method on the base class, accept the base class so you can serve as many clients as possible.

On the flip side, it’s good to return very specific types for similar kinds of reasoning. If I have a “GetDog()” method that instantiates and returns a Dog, why pretend that it’s a general Animal? I mean, it’s always going to be a Dog anyway, so why force my clients that are interested in Dog to take an Animal and cast it? I’ve blogged previously about what I think of casting. Be specific. If your clients want it to be an animal, they can just declare the variable to which they’re assigning the return value as Animal.

So, with this rule of thumb in mind, it would suggest that returning lists is a good idea when you’re definitely going to return a list. If your implementation instantiates a list and returns that list, with no possibility of it being anything else, then you might want to return a list. Well, unless…

Understanding the Significance of Interfaces

A counter-consideration here is “am I programming to an interface or in a simple concrete type.” Why does this matter? Well, it can push back on what I mentioned in the last section. If I’m programming a class called “RandomNumberProvider” with a method “GetMeABunchOfNumbers()” that creates a list, adds a bunch of random numbers to it, and returns that list, then I should probably return List<int>. But what if I’m designing an interface called IProvideNumbers? Now there is no concrete implementation — no knowledge that what I’m returning is going to be implemented as List everywhere. I’m defining an abstraction, so perhaps I want to leave my options open. Sure RandomNumberProvider that implements the interface only uses a list. But how do I know I won’t later want a second implementation called “DeferredExecutionNumberProvider” that only pops numbers as they’re iterated by clients?

As a TDD practitioner, I find myself programming to interfaces. A lot. And so, I often find myself thinking, what are the postconditions and abilities I want to guarantee to clients across the board? This isn’t necessarily, itself, a by-product of TDD, but of programming to interfaces. And, with programming to interfaces, specifics can bite you at times. Interfaces are meant to allow flexibility and future-proofing, so getting really detailed in what you supply can tie your hands. If I promise only an IEnumerable, I can later define implementers that do all sorts of interesting things, but if I promise an IList, a lot of that flexibility (such as deferred execution schemes) go out the window.

The Client’s Burden

An interesting way to evaluate some of these tradeoffs is to contemplate what your client’s pain points might be if we guess wrong. Let’s say we go with IEnumerable as a return type but the client really just wants a IList (or even just List). How bad is the client’s burden? Well, if client only wants to access the objects, it can just awkwardly append .ToList() to the end of each call to the method and have exactly what it wants. If the client wants to modify the state of the grouping (e.g. put the items in a different order and have you cooperate), it’s pretty hosed and can’t really use your services. However, that latter case is addressed by my “when a list is a no brainer” section — if your clients want to do that, you need not to give then IEnumerable.

What about the flip side? If the client really wants an IEnumerable and you give them a list? Most likely they want IEnumerable for deferred execution purposes, and you will fail at that. There may be other reasons I’m not thinking of off the top, but it seems that erring when client wants an enumerable is kind of a deal-breaker for your code being useful.

Ugh, so what should I do?!?

Clear as mud? Well, problem is, it’s a complicated subject and I can only offer you my opinion by way of heuristics (unless you want to send me code or gists, and then I can offer concrete opinions and I’m actually happy to do that). At the broadest level, you should ask yourself what your client is going to be doing with the thing that you return and try to accommodate that. At the next broadest level, you should think to yourself, “do I want to provide the client a feature-rich experience at the cost of later flexibility or do I want to provide the client a more sparse set of behavior guarantees so that I can control more implementation details?”

It also pays to think of the things you’re returning in terms of what they should do (or have done to them), rather than what they are. This is the line of thinking that gets you to ask questions like “will clients need to perform random accesses or sorts,” but it lets you go beyond simple heuristics when engaged in design and really get to the heart of things. Think of what needs to be done, and then go looking for the data type that represents the smallest superset of those things (or, write your own, if nothing seems to fit).

I’ll leave off with what I’ve noticed myself doing in my own code. More often than not, when I’m communicating between application layers I tend to use a lot of interfaces and deal a lot in IEnumerable. When I’m implementing code within a layer, particularly the GUI/presentation layer in which ordering is often important, I favor collections and lists. This is especially true if there is no interface seem between the collaborating components. In these scenarios I’m more inclined to follow the “return the most specific thing possible” heuristic rather than the “be flexible in an interface” heuristic.

Another thing that I do is try to minimize the amount of collections that I pass around an application. The most common use case for passing around bunches of things is collections of data transfer objects, such as some method like “GetCustomersWithFirstName(string firstName).” Clearly that’s going to return a bunch of things. But in other places, I try to make aggregation an internal implementation detail to a class. Command-Query Separation helps with this. If I can, I don’t ask you for a collection, do things to it and hand it back. Instead I say “do this to your collection.”

And finally, when in doubt and all else seems to be a toss-up, I tend to favor promising the least (thus favoring future flexibility). So if I really can’t make a compelling case one way or the other for any reason, I’ll just say “you’re getting an IEnumerable because that makes maintenance programming likely to be less painful later.”

By

I Give Up: Extroverted Barbarians at the Gates

Someone sent me a link to the video shown after this paragraph the other day, and I watched it. I then tweeted the link and sent it to a few of my coworkers because I figured it would make people laugh. It’s really funny, so give it a watch. But weirdly, I didn’t laugh. I watched it over and over again, mesmerized. I recognize that it’s funny and I find it funny, but I didn’t laugh.

This video is really a work of genius because it captures some incredible subtleties. There are two common archetypes captured nicely here in the form of the protagonist’s supposed allies: his boss and the project manager. I’ll give them names in their own sections below, along with the client characters. And then there are conversational tactics that bear mentioning.

This all revolves around a protagonist with whom any introverted person can identify. There’s nothing to indicate, per se, whether he’s introverted or extroverted, but the precision, the mannerisms, the posture — all of these scream “programmer” (or at least “engineer”) and so goes the association with introversion. The protagonist is the sole bulwark of sanity against a flood of idiocy, misunderstanding and general incompetence. You probably relate to him, having attended a meeting where all of the gathered C-levels and analysts thought you were being an obstructionist malingerer because you wouldn’t install Angry Birds on the meeting room’s television.

So who are the players?

Chamberlain

In a way, I liken the smarmy project manager, Walter, to former British prime minister, Neville Chamberlain, most remembered for his foreign policy of appeasement leading up to World War II in which he sought to dampen the aggression coming from the Axis powers by essentially “befriending” them. In this particular video, Chamberlain, the project manager, is presumably along to bridge the gap between the non-subject-matter-expert customers and the total-subject-matter-expert protagonist (and whose expertise makes the video eponymous). That’s not really why he’s there (though he doesn’t realize this), and I’ll get into that later as I’m describing tactics.

Chamberlain perceives that his best interests are served by simply agreeing to whatever is happening on the other side of the aisle, improvident though this may be. On some level, he’s probably aware that this strategy is stupid, but, hey, that’s a problem for later. He thinks his boss will skewer him if they don’t get the contract, so the fact that it’s going to be hard or impossible to deliver (what Expert is trying to tell him) just means he’ll later throw someone (i.e., Expert) under the bus.

Dilettante

The “design specialist,” Justine, is a mildly interesting character. She generally looks at Expert with some degree of respect and looks slightly uncomfortable when the rest of the characters make fun of Expert. At one point, to Expert’s delight, she even understands his point, and she visits him after the meeting out of genuine interest in the project and what is probably a “one pro to another” kind of overture. She’s the only character in the room that sees any value in Expert, and she probably recognizes that his subject matter knowledge exceeds hers and has value. If it were just her and Expert, she would probably listen attentively. I call her Dilettante because she seems to be the type of person you encounter with a bit of knowledge in a variety of fields and a genuine interest in improving.

Buffoon

The client’s boss is a classic MacLeod Clueless, and a simple idiot that isn’t very interesting. She’s the classic archetype of an over-promoted middle manager whose value is clearly wrapped up in company tenure. She spouts nonsensical jargon, torpedoes her firm’s own interests throughout the meeting, and serves up her position and her firm’s money for easy pickings by any MacLeod sociopath that happens along. She’s demanding something that she doesn’t understand in a way that makes no sense, and she’s willing to pay any huckster that comes along and sells her the magic beans she’s seeking.

Buffoon

Sociopath

Big Boss Man, to whom Chamberlain reports, is a classic MacLeod Sociopath. He likely has a fairly good handle on the situation and is of the opinion that the clients are idiots, but he has an intuitive understanding of the politics of the situation. Expert is flummoxed by the stupidity of the client proposal, and Chamberlain is simpering in an effort to show his boss his value as a diplomat, believing that the customer is always right and believing that Sociopath also believes that. Sociopath doesn’t. He knows the clients are idiots, and that Chamberlain is also kind of an idiot (for evidence of this, look at his expression at 6:14 where he clearly thinks the discussion of cats and birds as lines is dumb and simply ignores the client).

This doesn’t result in him rushing into defend Expert, however. That’s counter to his best interests, which I’ll address as a tactic, but he also finds Expert somewhat distasteful. Sociopath has navigated his way ably to money and power and a position atop the corporate hierarchy, but it is probably a slight annoyance to him that he may not be the smartest guy in the room. He knows that in Expert’s area of expertise, he’s nowhere near Expert, and while that’s fine, his inability to compare their relative intellectual worth across subject areas is a source of irritation.

Tactical Gamesmanship

So, the ostensible point of this meeting and no doubt many in which you’ve sat is to define the parameters for a project and then successfully launch that project. But, if you were to read the subconscious goals of the players, they would go something like this:

  • Chamberlain: I want to get the client to sign off no matter what, and I want Sociopath to think it was my heroics that made this happen.
  • Buffoon: I want to order people around and show off.
  • Sociopath: I want this to be over quickly so I don’t have to listen to Buffoon and Chamberlain.
  • Dilettante: I want to learn on the job without it being apparent that I’m doing so.
  • Expert: I want to define parameters for this project and successfully launch it.

Sociopath knows that getting Buffoon to agree to the project is a veritable certainty going into the meeting, and he knows that Chamberlain’s presence is valuable, but not for the reasons that Chamberlain thinks. Chamberlain thinks he’s there because he’s a “straight shooter/smooth talker” that “speaks Expert” but Sociopath just wants him there because he understands how to butter Buffoon’s bread — by causing Buffoon to think she’s won an exchange and humbled an Expert. He’s there because Sociopath knows he’ll team up with Buffoon to laugh at Expert. Dilettante is just window dressing.

So what are the tactics by which this happens? What makes this so cathartic for engineers and programmers to watch? Well, there are a number of things occurring here.

Seizing on the only part of an explanation you understand

There’s nothing to level the playing field quite like ignoring most of what someone talking to you says and seizing on some minor point. This has two advantageous for purveyors of rhetorical fallacy. First and foremost, it lets you pivot the discussion in a way that you find favorable, but secondly, it implies that your opponent has babbled on and on and over-complicated the matter (ala Reagan countering Carter — folksy and relatable countering egg-head). Near the beginning, Expert gives a detailed explanation, avoiding saying that it would be impossible to draw red line with green ink by talking about color blindness. It’s a long-winded, but technically accurate way of saying “that’s pretty much impossible,” and all Buffoon takes away from it is “so, in principle this is possible.”

Talking down to the expert because you don’t understand

When Expert asks Buffoon to clarify what she’s talking about with “transparent ink,” she patronizingly says she thought that he’d know what “transparent” means and that he’d better know what “red” means if he’s an Expert. A little later, she doesn’t understand what perpendicular means and when Expert accidentally exposes that, she blames him for not understanding her nonsense. It’s a relatively standard approach to strike first in blaming the other for a miscommunication between two parties, but it’s especially vitriolic in a case where the party in the driver’s seat is covering inadequacy.

Begging the question (and perverting the role of experts)

I’ve encountered this myself, in my travels, and it’s certainly on display here. People assume (from ignorance) that a certain outcome is possible/feasible, and then seek out an expert to make it happen. When the expert explains that they’re misguided or trying to do something ill-advised or impossible, they adopt the stance, “I thought you were an expert, and you’re telling me you can’t do this?” Chamberlain does this throughout the clip.

Dunning Kruger

This mostly comes from Sociopath and somewhat for show, but this is the tendency of those unskilled in a subject to assume that the subject is pretty simple and to generally devalue the knowledge of experts in that field. As more knowledge is acquired, so is respect for experts and humility. Sociopath dresses Expert down, particularly at the point where he says, “look, we’re not asking you to draw 20 lines — just 7.” Buffoon also does this once when she draws a triangle as an example of three perpendicular lines (“move — let me do it!”) Being the only Expert here and thoroughly outgunned and unaware of the real agenda, Expert is absolutely buried in an amplified echo chamber of Dunning Kruger.

Expert Introverts

These players and these tactics are painfully relatable. People in our line of work look at this ruefully and laugh because someone finally gets it and understands how silly the players seem to them. But introversion, lack of interest in office politics, and professional integrity are what hamstring us in such situations. I mean think of it this way — you cringe because you’re right there along with Expert, wanting these idiots to understand that red pens don’t draw green lines. You want to speak rationally to them and use analogies, diagrams and metaphors to make them see your point.

What you don’t do is turn the Dunning Kruger around on them and start telling them that they’re really going to need pure red lines if they want to maximize their verticals and strategize their synergies. You don’t tell them that kitten lines were so 2011. You don’t interrupt Chamberlain when he says “any fool can criticize” to say that you’re okay with the clients’ criticism and how he dare he call them fools. You don’t ask Chamberlain, if his title is “project manager,” why can’t he “manage” to define a clear spec.

You don’t do any of these things. Neither do I. Neither did Expert. Instead, we all do what he did in the end, which is to say, “sure, whatever buddy, I give up.” Extroverts extemporize and thrive in situations like this fueled with BS and beyond their control (though, Sociopath, who is controlling it, may be an introvert). We find ourselves at a loss for words, and utterly demoralized. Our credentials, our competence, and the validity of our very profession called into question, we bleakly resign ourselves to the madness and go home for a beer. We do that for a while, at least, and then, eventually, we Quit with a capital Q.

Perhaps that’s why I didn’t find myself laughing while watching this. Poor Anderson, the Expert, isn’t having an experience that he’ll submit to the Daily WTF and move on — he’s figuring out that his professional life is miserable. And the reason it’s miserable is because he’s realizing that expertise, ideas and results aren’t really the backbone of good business; in the land of the extroverts, egos and social capital are king.

Acknowledgements | Contact | About | Social Media