Stories about Software


Visualizing Your (Real) Architecture

Editorial note: I originally wrote this post for the NDepend blog.  Go check out the original at the NDepend site.  Take a look around at the other posts while you’re there as well — there’s a lot of good stuff to be had.

Diagrams of software architecture have a certain aesthetic appeal to them.  They usually consist of grayscale or muted pastel colors and nice, soft shapes with rounded edges.  The architects that make them assemble them into pleasing patterns and flowing structures such that they often resemble 7-layer cakes, pinwheels, or slalom courses.  With circles and ovals arranged neatly inside of rectangles connected by arrows, there is a certain, orderly beauty.  If you’re lucky, there will even be some fluffy clouds.

If you want to see an example, here’s one that has it all.  It’s even got a service bus.  Clearly, the battle for quality was over long before the first shots were ever fired.  After the initial conception of this thing, the mundane details of bringing the architecture to life would likely have been a simple matter of digital paint by numbers.  Implement an interface here, inherit from a framework class there, and Presto!  Instant operational beauty that functions as smoothly on servers as it does in the executive readout power point.

At least, that’s the plan.

Read More


Cutting Down on Code Telepathy

Let’s say that you have some public facing method as part of an API:

CustomerOrder is something that you don’t control but that you do have to use. Life is good, but then let’s say that a requirement comes in saying that orders can now be post-dated, so you need to modify your API somewhat, to something like this:

Great, but that was really painful because you learn that publishing changes to your public API is a real hassle for both yourself and for your users. After a lot of elbow grease and grumbling at the breaking change, though, things are stable once again. At least until a stakeholder with a lot of clout comes along and demands that it be possible to process orders through that method while noting that the order is actually a gift. You kick and scream, but to no avail. It has to go out and it has to hurt, and you’re powerless to stop it. Grumbling, you write the following code, trying at least to sneak it in as a non-breaking change:

But then you start reading and realize that life isn’t that simple and that you’re probably going to break your clients anyway. Fed up, you decide that you’re going to prevent yourself ever from being bothered by this again. You’ll write the API that stands the test of time:

Now, this can never be wrong. CustomerOrder can’t be touched, and the options dictionary can support any extensions that are requested of you from here forward. If changes need to be made, you can make them internally without publishing painful changes to the API. You have, fortunately, separated your concerns enough that you can simply deploy a new DLL that handles order processing, and any new values supplied by your clients can be handled. No more API changes — just a quick update, some testing, and an explanatory Word document sent to your client explaining how to use the thing. Here’s the first one:

There. A flexible API and the whole “is gift” thing neatly handled. If they specify that it’s a gift, you handle that. If they specify that it isn’t or just don’t add that option at all, then you treat those equally as the default case. Important stakeholder satisfied, and you won’t be bothered with nasty publications. So, all good, right?

Flexibility, but at what cost?

I’m guessing that, at a visceral level, your reaction to this sequence of events is probably to cringe a little, even if you’re not sure why. Maybe it’s the clunky use of a collection type instead of something slicker. Maybe it’s the (original) passing of a Boolean to the method. Perhaps it’s simply to demand to know why CustomerOrder is inviolate or why we couldn’t work to an order interface or at least define an inheritor. Maybe “options” reminds you of ViewState.

But, whatever it is, doesn’t defining a system boundary that doesn’t need to change seem like a worthwhile goal? Doesn’t it make sense to etch painful boundaries in stone so that all parties can rely on them without painful integration? And if you’re going to go that route, doesn’t it make sense to build in as much flexibility as possible so that all parties can continue to innovate?

Well, that brings me to the thing that makes me wince about this approach. I’m not a fan of shying away from the pain of “icky publish/integration” instead of going with “if it hurts, do it more and get better at it.” That shying away doesn’t make me wince in and of itself, but it does seem like the wrong turn at a fork in the road to what does make me wince, which is the irony of this ‘flexible’ approach. The idea in doing it this way is essentially to say, “okay, publishing sucks, so let’s lock down the integration point so that all parties can work independently, but let’s also make sure that we’re future proof so we can add functionality later.” Or, tl;dr, “minimize multi-party integration coordination with hyper-flexible API.”

So where’s the irony? Well, how about the fact that any new runtime-bound additions to “options” require an insane amount of coordination between the parties? You’re now more coupled than ever! For instance, let’s say that we want to add a “gift wrap” option. How does that go? Well, first I would have to implement the functionality in the code. Then, I’d have to test and deploy my changes to the server, but that’s only the beginning. From there, I need to inform you what magic string to use, and probably to publish a Word document with an example, since it’s easy to get this wrong. Then, once you have that document, I have to go through my logs and troubleshoot to discover that, “oh yeah, see that — you’re passing us ‘shouldGiftwrap’ when it should really be ‘shouldGiftWrap’ with a capital W.” And if I ever change it, by accident or on purpose? You’ll keep compiling and running, and everything will be normal except that, from your perspective, gift wrapping will just quietly stop working. How much pain have we saved in the end with this non-discoverable, counter-intuitive, but ‘flexible’ setup? Wouldn’t it be better not to get cute and just make publishing a more routine, friction-free experience?

The take-away that I’d offer here is to consider something about your code and your software that you may not previously have considered. It’s relatively easy to check your code for simple defects and even to write it in such a way to minimize things like duplication and code churn. We’re good at figuring out how not to have to keep doing the same thing over and over as well and to simplify. Those are all good practices. But the new thing I’d ask you to consider is “how much out of band knowledge does this require between parties?”

It could be a simple scenario like this, with a public facing API. Or, maybe it’s an internal integration point between your team and another team. But maybe it’s even just the interaction surface between two modules, or even classes, within your code base. Do both parties need to understand something that’s not part of the method signatures and general interaction between these entities? Are you passing around magic numbers? Are you relying on the same implicit assumptions in both places? Are there things you’re communicating through a means other than the actual interactions or else just not communicating at all? If so, I suggest you do a mental exercise to ask yourself what would be required to eliminate that out of band communication. Otherwise, today’s clever ideas become tomorrow’s maintenance nightmares.


Avoiding the Perfect Design

One of the peculiar ironies that I’ve discovered by watching the way a lot of different software shops work is that the most intense moments of exuberance about software seem to occur in places where software development happens at glacial speeds. If you walk into an agile shop or a startup or some kind of dink-and-dunk place that bangs out little CRUD apps, you’ll hear things like, “hey, a user said she thought it’d be cool if she could search her order history by purchase type, so let’s throw that in and see how it goes.” If it goes insanely well, there may be celebrations and congratulations and even bonuses changing hands which, to be sure, makes people happy. But their happiness is Mercury next to the blazing Sun of an ivory tower architect describing what a system SHALL do.

“There will be an enterprise service bus. That almost goes without saying. The presentation tier and the business tier will be entirely independent of one another, and literally any sort of pluggable module that you dream up as a client can communicate with any sort of rules engine embedded within the business tier. Neither one will EVER know about the other’s existence. The presentation layer collaborators are like Schrodinger and the decision engines are like the cat!

And the clients. Oh yes, there will be clients. The main presentation tier client is a mobile staging environment that will be consumed by Android, iOS, Windows Phone, Blackberry, and even some modified Motorolla walkie-talkies. On top of the mobile staging environment will be a service adapter that makes it so that clients don’t need to worry about whether they’re using SOAP or REST or whatever comes next. All of those implementations will hide behind the interface. And that’s just the mobile space. There are more layers and subtleties in the browser and desktop spaces, since both of those types of clients may be SPAs, other thick clients, thin clients, or just leaf nodes.

Wait, wait, wait, I’m not finished. I haven’t even told you about the persistence factories yet and my method for getting around the CAP theorem. The performance will be sublime. We’re talking picoseconds. We’re going to be using dynamically generated linear programming algorithms to load balance the vertical requests among the tiers, and we’re going to take a page out of the quantum computing book to introduce a new kind of three state boolean… oh, sorry, you had a question?”

“Uh, yeah. Why? I mean, who is going to use this thing and what do they want with it?”

“Everyone. For everything. Forever.”

You back out slowly as the gleam in his eye turns slightly worrisome and he starts talking about the five year plan that involves this thing, let’s call it HAL, achieving sentience, bringing humankind to the cusp of the singularity, and uploading the consciousnesses of all network and enterprise architects.


Like I said, the Sun to your Mercury. Has your puny startup ever passed the Turing Test? Well, his system has… as spec’ed out in a document management system with 8,430 pages of design documents and a Visio diagram that’s rumored to have similar effects to the Ark of the Covenant. And that, my friends, is why I think that a failing ATDD scenario should be the absolute first thing anyone who says, “I want to get into programming” learns to do.

Now to justify that whiplash-inducing segue. I wrote a book about unit testing in which I counseled complete initiates to automated testing to forgo TDD and settle for understanding the mechanics of automated tests and test runners first before making the leap. I stand by that advice, but I do so because I think that there is a subtle flaw to the way that most people currently get started down the programming path.

I was watching a Pluralsight course about NUnit to brush up on their latest and greatest assertion semantics, and the examples were really well done. In particular, there was a series of assertions oriented around a rudimentary concept of a role playing game with enumerations of weapons, randomization of events, and hit points. This theme exercised concepts like ranges, variance, collection cardinality, etc and it did so in a way that lent itself to an easy mental model. The approach was very much what mine would have been as well (I wouldn’t have come at this with TDD because there’d have been a lot of ‘downtime’ writing the production code as opposed to just showing the assert methods).

Nevertheless, it’s been a while since I’ve watched someone write tests against a pre-baked system when they weren’t characterization tests in a legacy rescue, and the experience was sort of jarring. I couldn’t help but think, “I wouldn’t want to write these tests if I were in his position — why bother when the code is already done?” Weird as it sounds from a big advocate of testing, writing tests after you’ve completed your implementation feels like a serious case of “going through the motions” in the same way that developers fill out random “SDLC” artifacts for no other purpose than to get PMPs to leave them alone.

And that’s where the connection to the singularity architect comes in. One of the really nice, but subtle perks of the TDD (especially ATDD) approach is that it forces you to define exit criteria before you start doing things. For instance, “I know I’ll be done with this development effort when my user can search her order history by purchase type.” Awesome — you’re well on your way because you’ve (presumably) agreed with stakeholders ahead of time when you can stop coding and declare victory. The next thing is to prove it, and you can approach this in the same way that you might approach fixing a leaking pipe under your sink. Turn the water on, observe the leak, turn the water off, fix the leak, turn the water back on, observe that there is no leak. Done.

In the case of the search, you write a client call to your web service that supplies a “purchase type” parameter and you say that you’re done when you get a known result set back, instead of the current error message: “I do not understand this ‘purchase type’ nonsense you’ve sent — 400 for you!” Then you scurry off to code, and you just keep going until that test that you’ve written turns green and all of the other ones stay green. There. Done, and you can prove it. Ship it.

Our poor architect never knows when he’s done (and we know he’ll never be done). The origin of this Sisyphean struggle started with hobby programming or a CS degree or something. It started with unbounded goals and the kinds of open-ended tasks that allow hobbyists to grow and students to excel. Alright, you’ve got the A, but try to play with it. See if you can make it faster. Try adding features to it. Extra credit! Sky’s the limit! At some point, a cultural norm emerges that says it’s more about the journey than the destination. And then you don’t rise through the ranks by automating for the sake of solving people’s problems but rather by building ever-more impressive juggernauts, leveraging the latest frameworks, instrumented with the most analytics, and optimized to run in O(Planck Time).

I really would like to see initiates to the industry learn to set achievable (but slightly uncomfortable) goals with a notion of value and then reach them. Set a beneficial goal, reach it, rinse, repeat. The goal could be “I want to learn Ruby and I’ll consider a utility that sorts picture files to be a good first step.” You’re adding to your skill set as a developer and you have an exit criteria It could be something for a personal project, a pro-bono client, or for pay. But tie it back to an outcome and assess whether that outcome is worthwhile. This approach will prevent you from shaving microseconds off of an app that runs overnight on a headless server and it will prevent you from introducing random complexity and dependency to an app because you wanted to learn SnazzyButPointless.js. True, the approach will stop you from ever delighting in design documents that promise the birth of true artificial intelligence, but it will also prevent you from experiencing the dejection when you realize it ain’t ever gonna happen.


Encapsulation vs Inversion of Control

This is a post that I’ve had in my drafts folder for nearly two years. Well, I should say I’ve had the title and a few haphazard notes in my drafts folder for nearly two years; I’m writing the actual post right now. The reason I’ve had it sitting around for so long is twofold: (1) it’s sort of a subjective, tricky topic and (2) I’ve struggled to find a concrete stand to take. But, I think it’s relatively important, so I’ve always vowed to circle back to it and, well, here we are.

The draft was first conceived when I’d given a presentation about testability and inversion of control — specifically, using dependency injection via constructors and setters to achieve these ends. I talked, among other things about the Open/Closed Principle, and how this allows modifications to system behavior that favor adding over editing code. The idea here is that we can achieve new functionality with a minimum of violence to the code base and in a way for which it is easy to write unit tests. Everyone wins, right?

Well, not everyone was drinking the Kool-Aid. I fielded a question about encapsulation that, at the time, I hadn’t prepared to answer. “Doesn’t this completely violate encapsulation?” I was a little poleaxed, and sputtered out an answer off the cuff, saying basically, “well, not completely….” I mean, if you’ve written a class that takes ILogger in its constructor and uses it for logging, I control the logger implementation but you control when and how it is used. So, you encapsulate the logger’s usage but not its implementation and this stands in contrast to what would happen if you instantiated your own logger or implemented it yourself — you would encapsulate everything and nothing would be up to me as a client of your code. Certainly, you have more encapsulation. So, I finished: “…not completely…. but who cares?!?” And that was the end of the discussion since we were out of time anyway.


I was never really satisfied with that answer but, as Creedence Clearwater says, “time and tears went by, and I collected dust.” When I thought back to that conversation, I would think to myself that encapsulation was passe in the same way that deep inheritance hierarchies were passe. I mean, sure, I learned that encapsulation was one of the four cornerstone principles of OOP, but so is inheritance, and that’s kind of going away with “favor composition over inheritance.” So, hey, “favor dependency injection over encapsulation.” Right? Still, I didn’t find this entirely satisfying — just good enough not to really occupy much of a place in my mind.

But then I remember a bit of a brouhaha last year over a Stack Overflow question. The question itself wasn’t especially remarkable (and was relatively quickly closed), but compiler author and programming legend Eric Lippert dropped by to say “DI is basically a bad idea.” To elaborate, he said:

There is no killer argument for DI because DI is basically a bad idea. The idea of DI is that you take what ought to be implementation details of a class and then allow the user of the class to determine those implementation details. This means that the author of the class no longer has control over the correctness or performance or reliability of the class; that control is put into the hands of the caller, who does not know enough about the internal implementation details of the class to make a good choice.

I was floored. Here we have one of the key authors of the C# compiler saying that the “D” in the SOLID principles was a “bad idea.” I would have dismissed it as blasphemy if (1) I were the sort to adopt approaches based on dogma and (2) he hadn’t helped author at least 4 more compilers than I have. And, while I didn’t suddenly rip the IoC containers out of my projects and instantiate everything inside constructors, I did revisit this topic in terms of my thoughts.

Maybe encapsulation, in the information hiding sense, isn’t so passe. And maybe DI isn’t a magic bullet. But why not? What’s wrong with the author of a class ceding control over some aspects of its behavior by allowing collaboration? And, isn’t any method parameter technically a form of DI, if we’re going to be pedantic about it?

The more I thought about it, the more I started to see competing and interesting use cases. Or, I should say, the more I started to think what class authors are telling their collaborators by using each of these techniques:

Encapsulation: “Don’t worry — I got this.”
Dependency Injection: “Don’t worry — if this doesn’t work, you can always change it.”

So, let’s say that you’re writing a Mars Rover or maybe a compiler or something. The attitude that you’re going to bring to that project is one in which correctness, performance and reliability are all incredibly important because you have to get it right and there’s little room for error. As such, you’re likely going to adopt implementation preference of “I’m going to make absolutely sure that nothing can go wrong with my code.”

But let’s say you’re writing a line of business app for Initrode Inc and the main project stakeholder is fickle, scatterbrained, and indecisive. Then you’re going to have an attitude in which ease and rapidity of system changes is incredibly important because you have to change it fast. As such, you’re likely to adopt an implementation preference of “I’m going to make absolutely sure that changing this without blowing everything up is easy.”

There’s bound to be somewhat of an inverse relationship between flexibility and correctness. As a classic example, a common criticism of Apple’s “walled garden” approach was that it was so rigid, while a common praise of the same was how well it worked. So I guess my take-away from this is that Dependency Injection and, more broadly, Inversion of Control, is not automatically desirable, but I also don’t think I can get to Eric’s take that it’s “basically a bad idea,” either. It’s simply an exchange of “more likely to be correct now” for “more likely to be correct later.” And in the generally agile world in which I live and with the kind of applications that I write, “later” tends to give more value.

Uncle Bob Martin once said, I believe in his Clean Coders video series, that (paraphrased) the second most important characteristic of good software is that it meet the customer requirements. The most important characteristic is that it be easy to change. Reason being, if a system is correct today but rigid, it will be wrong tomorrow when the customer wants changes. If the system is wrong today but flexible, it’s easy to make it right tomorrow. It may not be perfect, but I like DI because I need to be right tomorrow.


Notes on Writing Discoverable Framework Code

A while back, I found myself plugged into working on a rather large .NET project. The project had been going on for a while, and I was helping out only in terms of implementation manpower; architectural decisions had long since been made. And, in fact, some of the architectural decisions had pre-dated the project altogether. There was a “framework” that these guys were using for the purpose of rapid implementations. The idea was that developers could sort of slap this framework into place and make a few tweaks to have new applications rather than having to take more time to write the code. This wasn’t especially effective, but that was the goal.

As I used (and kind of battled with) this framework, something bothered me about it that I couldn’t put my finger on at first. But as I worked with it, I started to figure it out. The framework wasn’t discoverable in the slightest and thus fought against its own goal of rapid ramp-up and development. I actually put a placeholder in my drafts folder all of those moons ago wherein I made notes about how to make a framework discoverable. I never wound up bothering to make suggestions for improvements to that framework since, luckily, it was the last time I had to use it. But I’m trying to reduce the number of unpublished drafts in my folder, so I figured I’d offer up the suggestions to the world at large so that maybe some of you reading can get some value out of them.

What Do You Mean, Discoverable?

In the last paragraph I used a word without defining it, so I’ll correct that here. To understand what I mean, ask yourself the question, “how do developers discover how to use code?” 10 years ago, the most common answer to that question would have been “read the documentation.” Ah, the bad old days. I remember doing some Linux kernel programming using a library called RTAI, and I had these binders full of documentation on the API. They described in detail what each of the hundreds of parameters for dozens of functions were and what they meant. You certainly couldn’t tell from the names of these things — they were abbreviated, cryptic, and unclear.

The API itself lent few obvious clues about how to use it, and documentation was necessary for understanding. This is an API that is not discoverable. Discoverability is the degree to which you can figure out how to use something simply by playing with and examining it. For instance, if some API had a method called “Connect_To_The_SQL_Server_Database_Using_The_Following_Connection_String(string connection_string)”, you’d be dealing with something highly discoverable; you know how to use this without doing anything but looking at the code.

Developers have a variety of strategies for discovering how to use code. Reading documentation is probably the most traditional and established method, and one with which most experienced developers are familiar. However, let’s be frank — it’s also kind of a bummer. When you’re coding, you just want to write code via experimentation, trial and error. Having to lug out a big binder and pore through it, parsing jargon-heavy text is definitely a fly in the ointment. And while I’ve seen a lot of people express the sentiment, “you have to fight through the burn or you’re not a real programmer,” the fact of the matter is that when something is drudgery, attention spans wander and productivity decreases. So while reading documentation is the most established way of discovering code, it’s also the least effective, typically.

By contrast, the most effective manner of code discovery is use. Developers start using an API in whatever way seems most intuitive to them, typically using tools like Intellisense or word completion. They have some class they’ve declared an instance of in their IDE and they type the variable name and then a period and then they see what the auto-completion tool gives them by way of options for methods. There are other ways for developers to discover APIs as well, such as reading the code when it’s available, pairing with the code’s author, discussing the code with others, etc. But by and large, discovery through experimentation and clarity of purpose is most effective.

So the important question for architects, API authors, and framework providers thus becomes “how do I make my stuff discoverable for the developers that will use it?” Here are some suggestions.

Favor composition over inheritance

Inheritance is one of the main facets of polymorphism and boy oh boy was it popular in the late 1990’s and early 2000’s, especially in Java-land. I recall working on and with inheritance hierarchies that were 10 deep before any useful functionality emerged. Inheritance was one of the best ways at the time to avoid code duplication, but we, as an industry, learned the ugly downside of this approach: confusing non-discoverability.

There is nothing quite like getting a bug report that your code isn’t working and then taking a look to see what went wrong. You look at the source control history and see no record of any change that would explain the bug. None of the classes that your class is using have changed. Is it a bug in the GUI technology or database driver or something? Nope. Turns out, someone checked in a change 3 levels above you in the inheritance hierarchy and suddenly your class behaves differently. /Sigh.

When you use composition (the practice where your class uses other classes or implements interfaces), this sort of mysterious behavior in your classes is eliminated. If your class’s behavior has changed and you haven’t changed it, you know that one of the things that you’re using must have been changed. Eliminating or minimizing inheritance gets rid of a source of confusing possible changes.

Favor few parameters over many

This is relatively straightforward, but avoid using a lot of parameters with your methods. The fewer the parameters, the fewer the amount of things that callers of your methods have to understand and remember. This is a “Golden Rule” kind of situation. When you find the method you want to use by name, but it takes 12 arguments, do you think to yourself, “perfect — just what I want!” or do you think to yourself, “ugh, why me?” Well, users of your API/framework are no different. Consolidate the parameters you require into objects if need be — this will still be easier to understand for clients of your work.

Be judicious with overload methods

Method overloads are a powerful tool, but use them sparingly. It’s nice to offer your clients options, but it’s easy to offer them too many options. Applying the “golden rule” concept from the previous section, imagine if your code autocomplete tool shows you the method that you want, but there are 20 different parameter lists. Ugh. Which one to use? This isn’t discoverable anymore because you’re going to start googling or reading documentation or something. If there’s only one option, you’re simply going to use it and move on.

Favor early binding over late binding

One of the nastiest facets of the framework that I mentioned in the introductory section of the post was rampant late binding. Late binding is a scheme where relationships are resolved at run time rather than compile time. A common example of late binding would be a scheme that uses reflection to resolve names of fields in code to strings and to sync them with values on a form. So, if you have a Customer object with a “Name” property, you use reflection to get the string “Name” and look for a matching string on the form to which to write that value. Contrast this with a scheme where you simply have a strongly typed form object and you assign a value to its “Name” property. In the former scheme, mis-typing the property name results in a confusing runtime exception whereas in the latter scenario, the code won’t compile.

Programmers understand non-compiling. It’s one of the first feedback schemes you learn with a new language and you’ll very quickly understand why the compiler complains at you. So when something doesn’t compile, it’s quick and easy to figure this out and fix it. But if something fails weirdly at runtime as with a late binding scheme, it becomes hard to figure out what’s wrong. And, more importantly, it’s very frustrating. Part of discoverability is that it’s easy to understand why the right things work and the wrong things don’t. Late binding makes this understanding far more difficult for people working with the code.

Fail as early as possible

Very closely related is the concept of failing fast and early. An important concept for programmers is a tight feedback loop — a minimum of time between when you take an action and see the result of that action. A framework that creates negligible lapses between action and reaction is a discoverable one because users can quickly understand what works and what doesn’t. If there’s a lag between action and reaction, things are harder to figure out. This is a big problem with late binding, but it would also apply to a framework that’s slow running or slow to build as well.

Make screwing up impossible

Perhaps most important to discoverability is to make screwing up as hard as possible. Have clients of your code “fall into the pit of success.” This means taking whatever steps are possible to disallow doing the wrong thing. If a class Car in your framework requires an Engine in order to work, force people using your framework to pass Engine to Car’s constructor so that if they don’t, the code won’t compile. If they pass null, throw an exception. Contrast this with a behavior where the code makes passing in an engine optional and simply limps along and maybe throws a null reference exception somewhere if it’s missing. The latter approach is confusing — you’re communicating through your API that the Engine isn’t needed, and yet you’re failing everywhere it’s not present. You’re creating a minefield for your users rather than a pit of success, and consequently making your stuff really non-discoverable.