DaedTech

Stories about Software

By

Optimizing Proto-Geeks for Business

In a recent post, I talked about the importance of having Proto-Geeks in your software group rather than Loafers and the toxic impact of too many Loafers in the group. If you’ll recall, Proto-Geeks are automaters (in other words, developers) that are enthusiastic about new technologies, and Loafers are ones that don’t much care for it and have a purely utilitarian desire to automate. Loafers are interested in automating only to maximize their efficiency so that their benefit-to-effort ratio is maximized.

One thing that I mentioned only briefly in the last post was the idea that Loafers, in spite of having no love for new technologies or ideas, would have the best business sense. Where Proto-Geeks might go bounding off on unprofitable digressions simply for the sake of solving technological puzzles, Loafers will keep their eyes on the prize. I dismissed this as an issue by saying that Loafers were locally maximizing and self-interested rather than being concerned with the best interests of the group as a whole. In retrospect, I think that this is a bit of an oversimplification of both the motivations of the Loafer and the best way to navigate the balance between interest in new technology and conservative business sense.

Let me be clear: the best approach is to have as few Loafers as possible. So the real question is how to reign in Proto-Geeks and get them to have business sense. And I think the answer is a two-pronged one: use gamification strategies and do away with the tired notion that programmers don’t “know business” and that you stick them in a room, slide pizza under the door, and get software in return at some point.

PizzaUnderTheDoor

No More Project Managers

Typically, organizations create the role of “Project Manager” and “Software Developer” and, along with these roles, create the false dichotomy that the former “does business” and the latter “does technology.” Or, in the parlance from these posts, the former is a Fanboy and the latter is a Proto-Geek or a Loafer. There’s an interesting relationship between the project manager Fanboys and the Loafers, as compared with the PMs and the Proto-Geeks.

Specifically, PMs tend to like and identify with Loafers and be annoyed by Proto-Geeks. Why would Fanyboy identify with Loafer? That doesn’t seem to make a lot of sense, given that they’re in opposite quadrants. Fanboys root for technology while not being particularly adept or inclined toward automation on their own, while Loafers are leery of technology but reasonably good at automating things. So, what common ground do these archetypes find? Well, they meet in two ways:

  1. Because of the division of labor, PMs have to root for automation without doing it themselves, which means rooting for and depending on automaters.
  2. PMs that got their start in programming used to be or would have been Loafers.

If you consider the canonical question posed to Padiwan developers fresh out of school at an annual review, “do you want to be on the technical track to architect or the project management track,” you’re basically asking “are you a Proto-Geek or a Loafer?” If that seems like a harsh parallel to draw, think about what each response means: “I love technology and want to rise to the top of my craft,” versus, “programming is really just a means to a different end for me.” These are cut-and-dry responses for their respective quadrants.

So while you have one breed of Loafer that loosely corresponds to Lifer and just wants to bang out code and collect a paycheck, you have another breed that wants to bang out code just long enough to get a promotion and trade in Visual Studio and Eclipse for MS Project and a 120% booked Outlook calendar. Once that happens, it’s easy to trade in the reluctant automater card for enthusiastic tech Fanboy.

But a tension emerges from this dynamic. On the one hand, you have the people developing along the technical track, getting ahead because of their expertise in the craft for which everyone in the group is hired. On the other hand, you have a group that tends to underperform relatively in the same role, looking opportunistically for a less technical field of expertise and authority. The (incredibly cynical) logical conclusion of this dynamic is the “Dilbert Principle,” which holds that the least competent programmers will be promoted to project management where they can’t do as much damage to the software, consumed as they are with Gantt Charts and Six Sigma certifications and whatnot.

However cynical the Dilbert Principle might be and however harsh the “PM as disinterested programmer” characterization might be, there’s no altering the fact that a very real tension is born between the “tech track” and the “project management” track. This is exacerbated by the fact that “project manager” has “manager” in the title whereas “senior software engineer” or “architect” does not. Seem silly? Ask yourself what life might be like if Project Manager was renamed to the (more accurate) title “Status Reporter/Project Planner,” as observed on the Half Sigma blog:

It is often suggested that the most natural next move “up” is into project management. But the first problem with this situation is that project management sucks too. It doesn’t even deserve to have the word “management” in the title, because project management is akin to management as Naugahyde leather is to leather. Project planner and status reporter is the more correct title for this job. Once you take the word “manager” out of title, it loses a lot of its luster, doesn’t it? Everyone wants to be a manager, but few would want to be a project planner and I daresay no one would want to be a status reporter. Status reporting is generally the most hated activity of anyone who endeavors to do real work.

There’s little doubt that project managers are often de facto developer managers–generally not in the org chart, but certainly in terms of who is allowed by the organization to boss whom around. And so there tends to be a very real tension between the top technical talent and the top business-savvy talent. Ace Proto-Geeks resent being ordered around by their perceived inferiors who couldn’t hack it doing “real work” and ace Project Managers hate having to deal with prima donna programmers too stupid in the ways of office politics and business sense to put themselves on track toward real organizational power.

I submit that this is an entirely unnecessary wall to build. You can eliminate it by eliminating the “project manager” role from software groups. To take care of the “project planner” role responsible for perpetually inaccurate Gantt charts and other quixotic artifacts, just go agile. Involve the customer stakeholders directly with the development group in planning and prioritizing features. Really. Your devs are knowledge workers and bright people–you don’t need an entire role to run interference between them and customers. As for the “status reporter” role, come on. It’s 2013. If you don’t have an ALM tool that allows a C-level executive to pull up a snazzy, progress reporting chart automatically, you’re doin’ it wrong.

So the first step is to stop hiring Loafers with the specific intent that they’ll conflict with and reign in the Proto-Geeks. Running a business like US Congress isn’t a good idea. Split the meta-project/housekeeping tasks among the developers, rather than creating a clashing, non-technical position on the team for this purpose and pumping up its title in terms of office politics.

Gamify

So you’ve eliminated the Loafers, but now you need to get your Proto-Geeks to think about the bottom line once in a while. You can round them up and lecture them all you like, and if you take a hardline kind of approach, it might work after a fashion. But I really don’t recommend this because (1) Proto-Geeks are knowledge workers that are very good at gaming metrics, and (2) real talent is likely to leave. You need their buy-in and this requires you to partner with them instead of treating them like assembly line workers.

But the thing that runs a Proto-Geek’s motor is automating things using cool tools and technologies. At their core, they are puzzle solvers and tinkerers and they enjoy collecting a paycheck for figuring out how to get their geeky toys and playthings to solve problems in your organization’s problem domain. You need some way to collect and analyze the performance of machines and workers on your manufacturing floor? Dude, they can totally do that with the latest version of ASP MVC, and they can even use this new open-source site profiler to analyze the performance in realtime, and… what’s that? You only need the site and you don’t want them to get carried away with the profiler and whatever else they were about to say?

Well, just present this need to them as another problem within your domain and give them another puzzle to solve. Challenge them to build a site with as few dependencies as possible or tell them that they get to pick two and only two new toys for the next project or tell them that they can have as many toys as they want, but if they don’t make their deadlines, they don’t get any for the next project.

Another great trend from a business geek perspective is the way that cloud solutions like Azure and Salesforce work, where extra CPU cycles, memory usage, and disk space actually cost small amounts of money in real time. Nothing drives a Proto-Geek to do great things like this level of gamification where he knows–just knows–that after an all-nighter and some Mountain Dew, he can shave $40 per day here and $120 per day there.

These examples are really just off the cuff, so take them with a grain of salt, but the underlying message is important. You don’t need to hire people that are skeptical of any release of VBA after MS Access 2000 or who want to coast through programming at the entry level until they have enough seniority to be project managers in order to have team members focused on making the business profitable. You just need to have innovative and appropriate incentives in place. It’s the Proto-Geek’s job to get excited about technologies and problems and using technologies to solve problems. It’s management’s job to give them incentives that make them want to solve the right problems to ensure profitability.

By

I Screwed Up… You’re Welcome!

I’ve had trouble with the hosting of this site lately. As you may have noticed if you visit via the site instead of a feed reader, page load times had slowed to a crawl and sometimes the site was actually down. The hosting company, hostmonster, has provided me with good service over the last couple of years, both in terms of quality of hosting and responsiveness. I expected this to be no different a few days ago.

I called the first time I noticed substantial slowness, and I was told that it was some kind of aberration and everything was fine. A few days later, there was a problem again, and this time the source, I was told, was that there was a node down somewhere between me as a client and my site. Between these two incidents, my site had been slow. I shrugged it off and then noticed the next day that, yet again, my site was pretty much non-functional. Hoping for quicker turn-around and to see if anyone else was having issues, I mentioned this on Twitter and was quickly replied to by the Hostmonster help account with this message: “We see our admins quickly fixed an issue that came up on your server. It is already running now again.”

The only problem was that, in their hurry to congratulate themselves, they’d forgotten to see whether the problem had been fixed. It hadn’t. I called again and was told that everything looked fine to them, at which point I took to Twitter to solicit suggestions for alternative hosting options. A couple of days and a number of phone calls later, Hostmonster did work with me and the issue appears to be resolved. Apparently, a user sharing a physical machine with my site had been infested with some kind of SPAM or something and was causing resource thrashing and throttling of my site. They eventually migrated me to another machine. I haven’t yet decided what to do about hosting. I’m weighing the annoyance of my site being unresponsive for a few days against the hassle of migrating and the fact that there were some very helpful people there that I eventually got to deal with after the first few had told me what a great job they’d done at fixing my imaginary problem.

But pulling back from my experience and looking a little more critically, I think the point at which my reaction as a user went from “annoyed but understanding” to “pissed” holds some lessons for consultants or anyone who deals with end users in a support capacity. I base this on examining the situation and discovering what I found galling.

  1. Don’t publicly declare a user’s problem fixed without confirming the user’s agreement.
  2. Avoid minimizing the user’s problems with words like “little” to describe the scope of the problem or, as in my case, “[quick]” to describe the downtime.
  3. Make every reasonable attempt to reproduce a user’s problem and prove to them that you’re doing so.

I know that there’s a balance to be struck between avoiding unfair blame (i.e. marketing and spinning your product/service in the best light) and between providing your users with solutions. I also know that there is no shortage of user error to go around. But I think it’s really important to understand and empathize with users when they’re frustrated. Go on Twitter or some other public forum and read the really negative comments about software or technology services, and, at the core, you’ll see extremely frustrated users.

You’ll see users whose websites have been down for two days and who don’t know when they’ll be back up. You’ll see users who have been struggling for hours to figure out how to get “hello world” out of the stupid thing. You’ll see users who just needed to get some work done for a deadline when your horrible update rendered their software non-functional and caused them to catch flak at a meeting. And the list goes on. These are the people you’re dealing with. They’re not malicious or looking to drag your name through the mud; rather, they’re looking to vent. And that’s your opportunity to be a sympathetic ear and to turn an incensed customer into a loyal one.

How’s that, you ask? Well, as much as people hate it when they’re frustrated, they can be mollified by the sense that you see that they’re having a problem, you understand it, you can fix it, and that you’re sorry that it happened to them. “Geez, I’m really sorry that update made you miss your meeting–I’d be mad too.” I bet that’d take the edge off of most people’s anger. Probably more so than “that update ran fine for me here, and check out that awesome new font it gave you–you’re welcome!”

I’m going to make it a point to try to harness the memory of these experiences as a user to make me better and more conscientious as a software provider. I’d rather be remembered for the kind things that users say about me than the flattering things I say about myself when others are within earshot.

(I’m still weighing my options for the hosting situation and running tests to confirm the response times continue to be better than they were, but my thanks to Daniel and Bradyn at Hostmonster who were actually quite helpful following my initial period of frustration.)

By

In with the Proto-Geeks, Out with the Loafers

Technology Progressives vs Conservatives Reconsidered

When it comes to our relationship with technology, most people think in terms of the classic struggle between inventors that capture the imagination and laborers that don’t want to be imagination-captured right out of a steady paycheck. I think this is a rather obvious way to consider technology in our lives, but that there’s another dimension that sort of kicks in as an “aha” moment in much the same way that a line between “left” and “right” politically is trumped by the political compass quadrant system that also addresses libertarianism vs authoritarianism.

Towards that end, I propose quadrants related to technology. On one axis, there is the line from Luddite to Technologist. This is the line along which enjoyment of technology ranges when it comes to aesthetics and emotions. Luddites have a visceral dislike for technology, often because it makes them feel insecure or afraid (e.g. laborers in a factory who are about to be replaced by robots). Technologists have an emotional connection with technological advancement, generally because they find change and new possibilities exciting and intriguing (e.g. people that read about scientific advances, hobbyists that build robots, etc.).

On the other axis, there is a range from Laborer to Automater. Along this line, applied and practical interaction with technology varies. The Laborer will have little opportunity and/or motivation to use technology to automate day-to-day tasks (e.g. an artisan or a member of a think tank). Automaters embrace optimizations brought about through the use of technology and will typically seek to be as efficient as possible. People at this end of the spectrum value technology in a pragmatic sort of way, using it as well as process to optimize for time, cost, etc. People who use mobile devices to track appointments and tasks to the point of hyper-organization are good examples.

These two axes yield the following graph, with labeled quadrants.
TechGraph

The most obvious quadrants are the Luddite-Laborer and Technologist-Automater, as those are the roles that we think of as the Curmudgeon and the Proto-Geek, respectively. Curmudgeons hate and regard with suspicion anything that is newfangled and unfamiliar, while their polar opposite, Proto-Geeks, embrace things specifically because they’re new and flashy. All of the elements of the ethos of hard work and humble living (Protestant Work Ethic to the point of absurdity) versus endless, powerful novelty (decadence, convenience, playing God) come into play here, and this represents the linear spectrum that most envision about technology. The less frequently discussed quadrants are the ones that I’m calling, “Fanboy” and, “Loafer.”

“Fanboy” is the set of people that enjoy technology for its aesthetics and have a concept of tech-chic. These are people who will buy something like a cell phone for an accessory more so than for the various practical purposes of automating parts of their lives such as the aforementioned task tracking and organization. Non-technical gamers may also fall in this quadrant. The other less commonly considered quadrant is the Loafer, who prefers to automate things in life but has no love for new technologies and techniques. Contradictory as this may seem, it’s actually relatively common. And worse, it’s toxic to your software development group.

Inside the Mind of the Loafer

To understand the problem, it’s essential to understand what makes the Loafer tick. First of all, the Loafer isn’t a malingerer in the sense that he’ll shirk his responsibilities or do work that is of inferior quality. Rather, the Loafer seeks to optimize efficiency so that the value proposition of doing work is improved. In other words, if you task a Loafer with alphabetizing a list of 50,000 names, he’ll write a program to do it because that gets the job done more quickly. He’ll charge you the same rate regardless of methodology, though, so he’s automating specifically to allow himself more free time (or profit, as it were).

By way of contrast, consider the Proto-Geek’s response to this same task. She’d most likely blurt out, “That’s easy! I could do that in any programming language–you know what? I’ll do it in Ruby because that will give me a chance to play with Ruby. Cost? Oh, geez, I dunno. Maybe ten bucks…? It’s really no big deal.” The Proto-Geek is interested in solving the problem and leveraging cool new techs to do it. Profit/time optimizing is a secondary concern. (It is also interesting to consider the reaction of the other quadrants to this scenario. Curmudgeon would probably extol the virtues of doing it by hand on your mental acuity, and Fanboy would probably do it by hand while tweeting about how there should be something that did this automatically.)

Proto-Geeks view automation as having a high cool factor. This is in contrast to Loafer, who views it as having a high profit factor. But why is this a detriment to your group? After all, it stands to reason that Loafer has the best business sense. It seems as though it’d be important to have people on the team that chose the best tool for the job rather than because they were simply excited to use it. And that is true. The problem is that Automaters further down the chart toward Luddite are operating in a way to maximize their own personal efficiency and profit, rather than the group’s.

Most Loafers in a software group don’t particularly like what they do, but they know that if they do more of it faster, they get more money–or more time to, well, Loaf. So once they find a set of tools, languages, frameworks, etc. and become facile with them, they’re not only disinterested in new technologies–they’re actively resistant. Loafers want nothing more than to continue to solve all problems that come their way as quickly and with as little effort as possible, but with a locally-maximizing, greedy approach; investing a lot of extra effort in the shorter term to learn new ways of doing things more efficiently down the road is not appealing.

If you have one or two Loafers in your group, they can serve the useful purpose as natural conservatives that prevent Proto-Geeks from running buck-wild with new techs and frameworks. But this dynamic must be carefully monitored, like an inoculation, since the proportions can quickly become toxic. Tip the group’s dynamic a little too far into the Loafer quadrant and suddenly you have a bad case of “not invented here” and complete resistance to new ideas. There’s also frequently a strong correlation between Loafers and Expert Beginnerism; established Loafers often have a track record of some success with their approach and thus a powerful, built-in argument for inertia and the status quo. And finally, Loafers tend to kill enthusiasm for learning pretty quickly since presentation of new technologies is met with sarcasm, skepticism, and derision, making it essentially uncool to learn and embrace new things lest one be accused naivete or being some kind of shill.

If you’re in a position of authority within a group and recognize a Loafer dynamic, I’d suggest shaking things up to see if you can re-awaken or inspire a Technologist in some of them. Or perhaps you can infuse the group with some fresh-faced Proto-Geeks. If, on the other hand, you’re a powerless Proto-Geek in a Loafer group, you have a range of options from persuasion to proof-of-concept pitches to management to dusting off your resume. In either case, I think it’s important to recognize your surroundings and to try to seek out the quadrant in which you’re most comfortable.

By

Leading Software Teams of Humans

I’ve been working lately on a project where I handle the build and deployment processes, as well as supply code reviews and sort of serve as the steward of the code base and the person most responsible for it. Not too long ago, a junior developer delivered some changes to the code right before I had to roll out a deployment. I didn’t have time to review the changes or manually regression test with them in place (code behind and GUI, so the unit tests were of no use), so I rolled them back and did the deployment.

That night, I logged back on and reviewed the code that I had deleted prior to the build. I read through it carefully and made notes, and then I went back to the version currently checked in and implemented a simpler version of the fix checked in by the junior developer. That done, I drafted an email explaining why I had reverted the changes. I also included in it some praise for getting the defect fix right, my own solution requiring less code, and a few pointers as to what made my solution, in my opinion, preferable.

By the time I finished this and saved a draft to send out the next day, it was pretty late at night and I was tired. As I got ready for bed, I contemplated what had motivated me along this course of action when none the extra work I did was really required. Clearly I thought this course of action was important, but my motivations were mainly subconscious. I wanted to understand what guiding principles of leadership I might be abiding by here and hammer them out a bit. And I think I got it figured out.

  1. Justify and explain your decisions.
  2. Prove the merits of your approach–show, don’t tell.
  3. Let members of your team take risks, using you as a safety net.

Explain Yourself

As I discussed in a previous post, a leader/manager issuing orders “because I say so” is incrementally losing the respect of subordinates/team members. And that’s exactly what’s going on if you simply make decisions without getting input or explaining the reasoning after the fact–as would have been the case in my situation had I not sent the email. What you’re saying to people when you unilaterally take action isn’t really clear because it’s cryptic and terse. So they’re left to interpret for themselves. And interpret they will.

They’ll wonder if maybe you’re just a jerk or paranoid. They’ll assume that you think they’re not important enough to bother with an explanation. Perhaps they’ll surmise that you think they’re too stupid to understand the explanation. They’ll wonder if they did something wrong or if they could improve somehow.

And really, that last one is the biggest bummer of all. If you’re a team lead and you need to overturn a decision or go in a different direction for a valid reason, that’s a very teachable moment. If you sit with them and help them understand the rationale for your decision, you’re empowering them to mimic your decision-making next time around and improve. If you just undo all their work with no fanfare, they learn nothing except that fate (and the team lead) is cruel. Seriously–that’s the message. And there are few things more demoralizing to an engineer than feeling as though the feedback mechanism is capricious, unpredictable, and arbitrary.

Show, Don’t Tell

“Because I said so” costs you respect in another important way on a technical team, as well. Specifically, it threatens to undermine your techie cred. You may think you’re Guru and you may even be Guru, but if you don’t occasionally offer demonstrations, the team members will start to think that you’re just a loud-mouthed armchair quarterback suited only for criticism, yelling, and second-guessing.

ArmchairQuarterback

If you lead a team and you don’t know what you’re doing technically, you’ll obviously lose them. But if you’re leading a team and you have a cloud architecture “do as I say” approach at all times, the outcome will differ only in that you’ll lose them more slowly. You should do something to demonstrate your technical chops to them. This will go a long way. Rands does an excellent job of explaining this (and I pay a subtle homage to his ideas with my post’s title). You have to build something. You have to show them that you’re the leader for a reason other than because you’ve hung around the company for a while or you have friends in high places. Developers usually crave meritocracy, and flashing some chops is very likely to make them feel better about following your lead. And you get bonus points if the flashing of said chops saves them work/teaches them something/shows them something cool.

You Are the Safety Net

I worked in a group once that was permeated by fear. It was a cultural thing born out of the fact that the loudest group members were the most paranoid and their skepticism flirted dangerously with cynicism. Nobody could be trusted with changing the code base, so code reviews were mandatory, often angry affairs that treated would-be committers as dastardly saboteurs out to commit acts of digital terrorism. The result of this Orwellian vibe was that progress and change happened inordinately slowly. Group members were much more terrified of doing the wrong thing than of missing deadlines or failing to implement features, so progress crawled.

I could also tell that there were intense feelings of stress for some people working this way. The feeling that you have to constantly avoid making mistakes is crippling. It is the province of a culture of learned helplessness–get yelled at enough for making code changes that aren’t good enough and you’ll just resort to asking the yellers to sit with you and show you exactly what to do every time. You’ll take no risks, run no experiments, and come to grips with no failures. You won’t improve–you’ll operate as a grunt and collect a paycheck.

As a team leader, this is a toxic environment if performance is important to you (and it might not be–some people just like being boss and having the luxury of not being judged on lack of performance). A stagnant, timid team isn’t going to have the autonomy necessary to handle the unexpected and to dream up new and innovative ways of doing things. In order for those things to happen, your team has to be comfortable and confident that mistakes and failed experiments will be viewed as valuable lessons learned rather than staging points for demerits, blame, and loss of political capital.

SafetyNet

So if someone on your team checks in code that breaks the build or slips into QA with problems or what-have-you, resist at all costs the impulse to get worked up, fire off angry emails, publicly shame them, or anything else like that. If you’re angry or annoyed, take some deep breaths and settle down. Now, go fix what they did. Shield them from consequences of their good-faith actions that might make them gun-shy in the future. Oh, don’t get me wrong–you should certainly sit them down later and explain to them what they did, what the problem was, and what fixing it involved. But whatever you do, don’t do it in such a way that makes them scared of coding, changing things, and tinkering.

Will this approach mean some late nights for you? Yep. Does it suck to have to stay late cleaning up someone else’s mess? Sure it does. But you’re the lead. Good leaders work hard and make the people around them better. They offer enthusiasm and encouragement and a pleasant environment in which people can learn and grow. They don’t punch out at five like clockwork and throw anyone that threatens their departure time under the bus. Leadership isn’t a matter of entitlement–it’s a position of responsibility.

By

Flyweight

Quick Information/Overview

Pattern Type Structural
Applicable Language/Framework Agnostic OOP
Pattern Source Gang of Four
Difficulty Easy

Image courtesy of dofactory

Up Front Definitions

  1. Client: Code that uses the flyweight implementation
  2. Concrete Flyweight: A class that actually implements the abstract class/interface flyweight.
  3. Extrinsic State: State that is not common to the flyweights and varies per instance.
  4. Flyweight: An interface that defines and exposes the extrinsic state.
  5. Flyweight Factory: Class used by clients to retrieve instances of the flyweights.
  6. Intrinsic State: State that is common across all flyweights of the same type.

The Problem

Most of these design patterns posts deal with compile-time problems brought about by painting oneself into a corner with design decisions. In other words, if you make certain missteps when designing a system, it becomes rigid and difficult to change in the future. Patterns such as the Factories, Composite, Decorator, etc., all address prevention of such missteps by isolating responsibilities and making change management simple. Today, the problem we’re going to look at is a runtime problem rather than a compile-time problem.

Let’s say that you’re asked to create a model that simulates the distribution of mail in the USA for a given day. You start by establishing some pretty reasonable business objects to represent the domain concepts in question: letters and postcards. Each of these things has certain properties you’re going to need to keep track of, such as dimensions and postage, so that you can report in aggregate on paper usage, revenues, etc. Here is your first crack at business objects:

Alright, awesome. Now, according to the USPS, there are 563 million mail pieces processed in the USA each day (as of 2011). So, let’s just use that as our loop counter, and off we’ll go:

Satisfied, you fire up the program to make sure everything here is in good working order–although, really, why bother? I mean, what could possibly go wrong? The program chugs along for a bit, and it strikes you that it’s taking a while and your desktop is starting to crawl. You fire up the process manager and see this:

Danger, Will Robinson

Yikes! That’s… troubling. You start to wonder if maybe you should intervene and stop the program, but Visual Studio isn’t being very responsive and you can’t even seem to get it to hit a breakpoint. Just as you’re contemplating drastic action, Visual Studio mercifully puts an end to things:

OutOfMemory

You’re not very close to getting all of the records in, so you decide to divide the original 563 million by 24 in order to model the mail being processed on an hourly basis, but that’s really all the more granular that you can do. You fire up the program again to run with 23,458,333 and hit out of memory once again.

So, What to Do?

If you look at your creation of letters (and presumably later, postcards) you should notice that there’s a fair bit of waste here. For every letter you create, you’re creating four decimals that are exactly the same for every letter, which means that you’re storing four unnecessary decimals for all but one of the letters you create, which in turn means that you’re storing somewhere in the neighborhood of two billion unnecessary decimal instances in memory. Whoah.

Let’s try a different approach. What we really seem to want here is 23 and a half million zip codes and a single letter. So let’s start off simply by building a list of that many zip codes and seeing what happens:

Woohoo! That runs, and for the first time we have something that doesn’t crash. But what we’re really looking to do is cycle through all of those mail pieces, printing out their zip, postage, and dimensions. Let’s start off by cheating:

That gets us what we want, but there’s a lot of ugliness. The client class has to know about the exact mechanics of the mail piece’s printing details, which is a no-no. It also has the responsibility for keeping track of the flyweight letter class. There are better candidates for both of these responsibilities. First of all, let’s move the mechanism for printing information into the mail piece class itself.

Notice that we’ve removed the settable “DestinationZip” and added a method that prints piece statistics, given a zip code. That allows us to simplify the client code:

Asking the letter object for its statistics definitely feels like a better approach. It’s nice to get that bit out of the client implementation, particularly because it will now work for postcards as well and any future mail pieces that we decide to implement. But we’re still worrying about instance management of the flyweights in the client class, which really doesn’t care about them. Let’s introduce a new class:

Here we have a public method called “GetPiece” to which we pass a key indicating what sort of piece we want. As far as client code goes, that’s all that matters. Under the hood, we’re managing the actual flyweights. If the dictionary doesn’t have key we’re interested in, we build a piece for that key. If it does, we simply return the flyweight from the hash. (Note that the scheme of indexing them by characters will scale, but isn’t really optimal at the moment–you could use an enumeration here, restrict the set of allowed keys with preconditions, or have a special default scheme.)

Let’s see what the client code looks like.

There. Now the client code doesn’t worry at all about keeping track of the flyweights or about how to format the statistics of the mail piece. It simply goes through a bunch of zip codes, creating random pieces for them, and printing out their details. (It could actually dispense with the list of zip codes, as implemented, but leaving them in at this point underscores the memory benefit of the flyweight pattern.)

There’s now some nice extensibility here as well. If we implemented flats or parcels, it’d be pretty easy to accommodate. You’d just add the appropriate class and then amend the factory, and you’d be off and running. When dealing with scales of this magnitude, memory management is always going to be a challenge, so using a pattern up front that reuses and consolidates common information will be a big help.

A More Official Explanation

According to dofactory, the purpose of the Flyweight pattern is:

Use sharing to support large numbers of fine-grained objects efficiently.

The Flyweight pattern requires that you take the objects whose information you want to share for scaling purposes and divide their state into two categories: intrinsic and extrinsic. Intrinsic state is the state that will be shared across all instances of the same type of object, and extrinsic state is the state that will be stored outside of the instance containing common information. The reason for this division is to allow as much commonality as possible to be stored in a single object. In a way, it is reminiscent of simple class inheritance where as much common functionality as possible is abstracted to the base class. The difference here is that a single instance stores common information and an external interest holds the differences.

To make the most of the pattern, flyweight creation and management is abstracted into one class (the factory), intrinsic state to another set of classes (the concrete flyweights), and extrinsic state management to a third class (the client). Separating these concerns allows the client to incur only the memory overhead of the part of an object that varies while still reaping all of the benefits of OOP and polymorphism. This is a powerful optimization.

Other Quick Examples

  1. The iconic example of the Flyweight pattern is to have flyweights represent characters in a word processing document, with the extrinsic state of position in the document.
  2. Another use is rendering in graphical applications with many similar shapes or patterns on the screen in configurable locations.
  3. It can also be used to represent some fungible commodity of which there are many in a video game, such as enemies, soldiers, scenery elements, etc.
  4. One of the more interesting and practical examples is string “interning” in C#

A Good Fit – When to Use

As the cost of memory and disk space continues to plummet, emphasis on this pattern is seeming to wane a bit. However, there will always be occasions in which resource management and optimization are crucial, and bottlenecks will always exist where even liberally purchased resources need to be used wisely. The flyweight pattern is a good fit any time you have redundant, common state in many objects that you’re creating. If you find yourself in this position, the pattern is really a no-brainer since it’s simple and costs very little to implement in terms of code complexity.

Square Peg, Round Hole – When Not to Use

This isn’t the kind of pattern where you’re likely to go out looking for excuses to try it out since it solves a very specific problem that is different from a lot of other patterns. I would exercise caution more when choosing extrinsic and intrinsic state in that you’re going to experience downstream pain if you label things “intrinsic” that actually can vary. Doing this threatens to break the pattern and the abstraction that you’ve created since modifying state that’s categorized as intrinsic will mean that your flyweights are no longer interchangeable and you’re going to start getting stuff wrong.

So this isn’t a good fit if you don’t have a good grasp yet on what should be extrinsic and what should be intrinsic. It’s also a bad fit if all state is extrinsic, as then there is no gain from the pattern. Another potential misuse that’s always worth mentioning with an optimization strategy is premature optimization. While I personally consider the elimination of duplication at runtime simply to be good design, I’d say that you might want to consider not bothering with the additional compile-time complexity if you’re not going to have very many objects. If, for instance, you were simulating twenty pieces of mail instead of tens of millions, it might not be worth the additional complexity.

So What? Why is this Better?

This one pretty much sells itself. In the example above, you can run the code without crashing or altering your data structures. Generally speaking, this is a pattern that helps you reduce your application’s memory footprint simply by creating a clever abstraction and shuffling the internal representation. If you can identify when there will be common information in volume, you can separate it from the unique information and make sure you’re being as efficient as possible.