Stories about Software


Agile Methodologies or Agile Software?

Over the last couple of months, I’ve been doing mostly management-y things, so I haven’t had a lot of trade craft driven motivations to pick Pluralsight videos to watch while jogging. In other words, I’m not coming up to speed on any language, framework, or methodology, so I’m engaging in undirected learning and observation. (I’m also shamelessly scouring other authors’ courses for ways to make my own courses better). That led me to watch this course about Agile fundamentals.

As I was watching and jogging, I started thinking about Agile Manifesto and the 14 years that have passed since its conception. “Agile” is undeniably here to stay and probably approaching “industry standard.” It’s become so commonplace, in fact, that it is an industry unto itself, containing training courses, conferences, seminars, certifications — the works. And this cottage industry around “getting Agile” has sparked a good bit of consternation and, frequently, derision. We as an industry, critics might say, got so good at planning poker and daily standups that we forgot about the relatively minor detail of making software. Martin Fowler coined a term, “flaccid Scrum” to describe this phenomenon wherein a team follows all of the mechanics of some Agile methodology (presumably Scrum) to the letter and still produces crap.

It’s no mystery how something like this could happen. You’ve probably seen it. The most common culprit is some “Waterfall” shop that decides it wants to “get Agile.” So the solution is to go out and get the coaches, the certifiers, the process experts, and the whole crew to teach everyone how to do all of the ceremonies. A few weeks or months, some hands on training, some seminars, and now the place is Agile. But, what actually happens is that they just do the same basic thing they’d been doing, more or less, but with an artificially reduced cycle time. Instead of shipping software every other year with a painful integration period, they now ship software quarterly, with the same painful integration period. They’re just doing waterfall on a scale of an eighth of the previous size. But with daily standups and retrospectives.

There may be somewhat more nuance to it in places, but it’s a common theme, this focus on the process instead of the “practices.” In fact, it’s so common that I believe the Software Craftsmanship Manifesto and subsequent movement was mainly a rallying cry to say, “hey, remember that stuff in Agile about TDD and pair programming and whatnot…? Instead of figuring out how to dominate Scrum until you’re its master, let’s do that stuff.” So, the Agile movement is born and essentially says, “let’s adopt short feedback cycles and good development practices, and here are some frameworks for that” and what eventually results is the next generation of software process fetishism (following on the heels of the “Rational Unified Process” and “CMM”).

That all played through my head pretty quickly, and what I really started to contemplate was “why?” Why did this happen? It’s not as if the original signatories of the manifesto were focused on process at the exclusion of practices by a long shot. So how did we get to the point where the practices became a second class citizen? And then, the beginnings of a hypothesis occurred to me, and so exists this post.

The Agile Manifesto starts off with “We are uncovering better
of developing software…” (emphasis mine). The frameworks for this type of development were and are referred to as “Agile Methodologies.” Subtly but very clearly, the thing we’re talking about here — Agile — is a process. Here were a bunch of guys who got together and said, “we’ve dumped a lot of the formalism and had good results and here’s how,” and, perversely, the only key phrase most of the industry heard was “here’s how.” So when the early adopted success became too impressive too ignore, the big boys with their big, IBM-ish processes brought in Agile Process People to say, “here’s a 600 page slide deck on exactly how to replace your formal, buttoned-up waterfall process with this new, somehow-eerily-similar, buttoned-up Agile process.” After all, companies that have historically tended to favor waterfall approaches tend to view software development as a mashup of building construction and assembly line pipelining, so their failure could only, possibly be caused by a poorly engineered process. They needed the software equivalent of an industrial engineer (a process coach) to come in and show them where to place the various machines and mindless drones in their employ responsible for the software. Clearly, the problem was doing design documents instead of writing story cards and putting Fibonacci numbers on them.

The Software Craftsmanship movement, I believe, stands as evidence to support what I’m saying here. It removes the emphasis altogether from process and places it, in very opinionated fashion, on the characteristics of the actual software: “not only working software, but also well-crafted software.” (emphasis theirs) I can’t speak exactly to what drove the creation of this document, but I suspect it was at least partially driven by the obsession with process instead of with actually writing software.


All of this leads me to wonder about something very idly. What if the Agile Manifesto, instead of talking about “uncovering better ways,” had spoken to the idea of “let’s create agile software?” In other words, forget about the process of doing this altogether, and let’s simply focus on the properties of the software… namely, that it’s agile. What if it had established a definition that agile software is software that should be able to be deployed within, say, a day? It’s software that anyone on the team can change without fear. It’s software that’s so readable that new team members can understand it almost immediately. And so on.

I think there’s a deep appeal to this concept. After all, one of the most annoying things to me and probably to a lot of you is having someone tell me how to solve a problem instead of what their problem is, when asking for help. And, really, software development methodologies/processes are perhaps the ultimate example of this. Do a requirements phase first, then a design phase, then an implementation phase, etc. Or, these days, write what the users want on story cards, have a grooming session with the product owner, convene the team for planning poker, etc. In both cases, what the person giving the direction is really saying is, “hey, I want you to produce software that caters to my needs,” but instead of saying that and specifying those needs, they’re telling you exactly how to operate. What if they just said, “it should be possible to issue changes to the software with the press of a button, it needs to be easy for new team members to come on board, I need to be able to have new features implemented without major architectural upheaval, etc?” In other words, what if they said, “I need agile software, and you figure out how to do that?”

I can’t possibly criticize the message that came out of that meeting of the minds and gave birth to the Agile Manifesto. These were people that bucked entrenched industry trends, gained traction, and pulled it off with incredible success. They changed how we conceive of software development and clearly for the better. And it’s entirely possible that any different phrasing would have made the message either too radical or too banal for widespread adoption. But I can’t help but wonder… what if the call had been for agile software rather than agile methods.


You Want an Estimate? Give Me Odds.

I was asked recently in a comment what I thought about the “No Estimates Movement.” In the interests of full disclosure, what I thought was, “I kinda remember that hashtag.” Well, almost. Coincidentally, when the comment was written to my blog, I had just recently seen a friend reading this post and had read part of it myself. That was actually the point at which I thought, “I remember that hashtag.”

That sounds kind of flippant, but that’s really all that I remember about it. I think it was something like early 2014 or late 2013 when I saw that term bandied about in my feed, and I went and read a bit about it, but it didn’t really stick with me. I’ve now gone back and read a bit about it, and I think that’s because it’s not really a marketing teaser of a term, but quite literally what it advertises. “Hey, let’s not make estimates.” My thought at the time was probably just, “yeah, that sounds good, and I already try to minimize the amount of BS I spew forth, so this isn’t really a big deal for me.” Reading back some time later, I’m looking for deeper meaning and not really finding it.

Oh, there are certainly some different and interesting opinions floating around, but it really seems to be more bike-sheddy squabbling than anything else. It’s arguments like, “I’m not saying you shouldn’t size cards in your backlog — just that you shouldn’t estimate your sprint velocity” or “You don’t need to estimate if all of your story cards are broken into small enough chunks to be ones.” Those seem sufficiently tactical that their wisdom probably doesn’t extend too far beyond a single team before flirting with unknowable speculation that’d be better verified with experiments than taken as wisdom.

The broader question, “should I provide speculative estimates about software completion timelines,” seems like one that should be answered most honestly with “not unless you’re giving me pretty good odds.” That probably seems like an odd answer, so let me elaborate. I’m a pretty knowledgeable football fan and each year I watch preseason games and form opinions about what will unfold in the regular season. I play fantasy football, and tend to do pretty well at that, actually placing in the money more often than not. That, sort of by definition, makes me better than average (at least for the leagues that I’m in). And yet, I make really terrible predictions about what’s going to happen during the season.


At the beginning of this season, for instance, I predicted that the Bears were going to win their division (may have been something of a homer pick, but there it is). The Bears. The 5-11 Bears, who were outscored by the Packers something like 84-3 in the first half of a game and who have proceeded to fire everyone in their organization. I’m a knowledgeable football fan, and I predicted that the Bears would be playing in January. I predicted this, but I didn’t bet on it. And, I wouldn’t have bet even money on it. If you’d have said to me, “predict this year’s NFC North Division winner,” I would have asked what odds you were giving on the Bears, and might have thrown down a $25 bet if you were giving 4:1 odds. I would have said, when asked to place that bet, “not unless you’re giving me pretty good odds.”

Like football, software is a field in which I also consider myself pretty knowledgeable. And, like football, if you ask me to bet on some specific outcome six months from now, you’d better offer me pretty good odds to get me to play a sucker’s game like that. It’d be fun to say that to some PMP asking you to estimate how long it would take you to make “our next gen mobile app.” “So, ballpark, what are we talking? Three months? Five?” Just look at him deadpan and say, “I’ll bite on 5 months if you give me 7:2 odds.” When he asks you what on earth you mean, just patiently explain that your estimate is 5 months, but if you actually manage to hit that number, he has to pay you 3.5 times the price you originally agreed on (or 3.5 times your salary if you’re at a product/service company, or maybe bump your equity by 3.5 times if it’s a startup).

See, here’s the thing. That’s how Vegas handles SWAGs, and Vegas does a pretty good job of profiting from the predictions racket. They don’t say, “hey, why don’t you tell us who’s going to win the Super Bowl, Erik, and we’ll just adjust our entire plan accordingly.”

So, “no estimates?” Yeah, ideally. But the thing is, people are going to ask for them anyway, and it’s not always practical to engage in a stoic refusal. You could politely refuse and describe the Cone of Uncertainty. Or you could point out that measuring sprint velocity with Scrum and extrapolating sized stories in the backlog is more of an empirically based approach. But those things and others like them tend not to hit home when you’re confronted with wheedling stakeholders looking to justify budgets or plans for trade shows. So, maybe when they ask you for this kind of estimate, tell them that you’ll give them their estimate when they tell you who is going to win next year’s super bowl so that you can bet your life savings on their guarantee estimate. When they blink at you dubiously, smile, and say, “exactly.”


In Devs We Trust

The idea of getting certified in “Scrum” or “Agile” is philosophically unsettling to me. It’s not so much that I think learning how experienced people execute these processes is a bad thing (I’ve even signed off on such a course for an employee’s career development aspirations), but rather the overtones of such a certification. Certifications are formalized recognition of program completion and endorsements of those that have done the completing. It’s like saying, “congratulations — you have rigorously followed the formalized, rigid process necessary to be agile!” Against the backdrop of the original agilistas rejecting button-up formalism for pragmatism and efficiency, this is sort of like a high school principal commissioning a “Responsible Pot Smoking and 3rd Period Ditching” course as part of the core curriculum.

The reason this is unsettling to me rather than comically ironic is that it supplies surprisingly fertile ground for re-growing all of the mistakes of the waterfall era by masking the true nature of those mistakes. The idea that it’s possible to nail down every last detail of a complex software system prior to writing it is cringe-worthy in its naivete (at least this side of a space shuttle), but it’s not really the core problem. The core problem is the idea that a software project can’t succeed without planning every detail prior to implementation.

This attitude belies a fundamental lack of trust in the people doing the implementation and, further, a suspicion that these people are incompetent malingerers. As an industry, we need to set aside a few high-wage “architects” to create a specification that will allow the drooling imbeciles, or “programmers,” to perform the mindless, labor-intensive task of writing software. It evokes classic imagery of pyramid construction — Pharaoh and his great builders have the vision, and they just need whips, the wheel, and the backs of a few tens of thousands of fungible, expendable drones to make it a reality. The fundamental problem with the waterfall approach isn’t that we want to plan out the pyramid but that we want to treat software developers like stone-haulers, as if implementing an efficient self-balancing tree were akin to pushing rocks.

Everything about this “classic” approach to software smacks of this fundamental lack of trust. Why plan everything out ahead of time in excruciating detail? Why come up with elaborate, painfully detailed processes and mind-numbingly detailed “design documents?” Why lay everything out in such a way that there is absolutely no room for creative license or spontaneous thinking? Why come up with CMMI level 5? You do this to prevent people from “getting any ideas” and thinking for themselves — you do it because you don’t trust them (and you probably don’t trust them because you approached hiring them the way you’d approach finding cut-rate car insurance).

If you go read the Agile Manifesto, you’ll see something that’s relatively vague, but underpinned by a consistent theme: “we work best when you trust us, collaborate with us and talk to us.” It begs producers and consumers to back away from heavy documentation, plan, and process in favor of adaptability, collaboration and thinking on the fly. It implicitly asks that consumers of software trust producers of software and rely on their discretion.

So fast forward to present time and a world where you can depend on a thriving cottage industry to come in and bless your organization by making you “Agile Certified.” Why would you do this? Why bring people in and have them tell your developers in specific, painful detail exactly how to conduct meetings and follow the processes that will allow them to be certified agile practitioners? Why spell out exactly how they can go about being agile? You do it because you still don’t trust them.

Scrum and XP are approaches that have merit because they generally consist of individual, specific practices and ideas that may help a team. Pair programming/peer review, test driven development, making problems visible, etc are potential tools in the tool belt of a solid software developer. Agile practices/processes have value not because they’re the secret sauce that’s going to allow you to deliver version 6 on time where versions 1 through 5 have been lawsuit-inviting nightmares, but rather they have value when and because they force you to trust the software developers to get things done, to try hard, to provide honest answers and, generally, to be professionals.


There is no secret sauce. There is no up-up-down-down-left-right-left-right-b-a-b-a-start (I honestly did that without looking it up). There is no achievement to unlock, no box to check, and no certification to buy. Organizations don’t get off that easily because what they need to do is much harder to quantify as a line item, objective, or KPI. Fortunately, hard as it is to package up and roll into an annual report, it’s really easy to explain. Organizations have to hire software developers that they trust, clear nonsense and obstacles out of their way, keep them happy, and then trust them to develop software. That’s it.

I’m not saying this is actually easy to execute. Refining a candidate selection process to the point where you bring in competent, hard-working candidates with a high degree of certainty is quite a difficult problem. Keeping them happy is no simpler. Setting them up for success just keeps increasing the difficulty level. And paying for all of this is really going to make you wince since it’s not possible to short-cut the system by bringing folks in cheap and then, real-quick, getting them certified in “you can trust this person to make good decisions about writing software.” Perhaps hardest of all, though, given our historical culture of micro-management and death-by-process, is actually doing the trusting. But it has to be done. It’s the only sane way forward.

If you want to get a piece of software written, find some software developers you respect and trust and task them with developing software for you. If they know what they’re doing, they’ll figure out how to validate and verify their work, how to evaluate and improve their process as they go, how to limit distractions and work in progress, how to know when they’ve delivered on what they promised, etc. They might do this by Scrum or by XP or by something that they invent on the fly, mixing and matching and innovating, but they’ll do it. They don’t need elaborate process or ceremony instruction and they certainly don’t need to be ‘certified'; they need you to trust them and let them work as they see fit.


Agile: False Hope and Real Promise

I went shopping for jeans last weekend, which was, as always, somewhat of an aggravating experience. It’s not just jeans, but, really, any wearable thing: shoes, work shirts, belts, even socks. The source of my irritation is the fact that I can’t simply replace items I have that are worn out because the thing you bought last time is always decommissioned. Strangely, major things like cars have better staying power than clothes. There’s a very simple use case missing: As a shoe wearer, given that I love a pair of shoes and thus wear them out in 2 years, I want to go back to the same store and buy the exact same pair of friggin’ shoes so that my life is not interrupted. And the software of life fails to perform.

The reason for this is the nebulous concept of fashion. Naturally, wearable manufacturers and retailers don’t want to be ‘stale’ so they churn things at a rate that prevents me from being pragmatic with my shopping and turns my trips into maddening treasure hunts instead of simple errands. So I contemplate fashion a bit with a natural bias toward annoyance, and this has informed a pretty cynical take on the subject. I have an ongoing hypothesis that the thing that drives fashion isn’t some guy sitting in Paris, saying “I’m going to make those idiots wear really tight jeans this year and then laugh at them.” Instead, its the shifting tectonic plates of poor people trying to look like rich people and rich people trying not to look like poor people.

By way of narrative, guy in Paris releases skinny jeans and charges $700 for them. They’re new and expensive, so rich people buy them so that they can be seen wearing new and expensive things — signaling their wealth and cosmopolitan tastes. Bulk clothing manufacturers start knocking off Paris designer and selling it to the masses for $39 a pair and they sell out like crazy as everyone wants to ape the A list celebrities and influencers wearing these things. Rich people get annoyed that they’re no longer distinguishable on sight and this demand drives Paris guy to dream up some new, weird thing that they can buy to look rich. Rinse, repeat.


This narrative isn’t limited to the arena of rich and poor people with clothes, however. It also extends to things like industry thought leadership and buzzwords. Crowd of cool kids starts doing something and succeeding/making money. They give it a name and everyone rushes to mimic their success, adopting this thing. Imitators adopt it en masse, bring mediocrity and misunderstanding to it, and the cool kids lament that everyone is getting it wrong and that it’s time for another industry shake-up. Queue the “Is {Insert Tech} Dead” link bait and search for the next great thing.

“Agile” is probably the poster child for this sort of thing in the development world. Back around the turn of the millennium, various industry thinkers were experimenting with ways to move software development away from software development practices that mimicked construction and physical engineering since the results produced by this traditional, “waterfall” approach were spotty at best. A group of representatives of some of these experiments came together to find common ground and the result was called, “The Agile Manifesto.” In the interceding 13 years, the industry has come to accept that the Agile approach is “the right thing” while agreeing that, by and large, no one does it right.

Off the top, I’ve heard the following expressed recently (paraphrased from my recollection):

  • Agile is dead.
  • Scrum is a scam that makes grunts stand while they deliver their status reports.
  • If XP doesn’t work it’s because you’re doing it wrong.
  • Agile is stupid because you really can’t work sanely without comprehensive documentation and specs.
  • Agile is a marketing gimmick to create phony demand for meaningless certifications.

And yet, you’d be hard-pressed to find any non-agile shop that didn’t look down shamefacedly and mumble “yeah, we do waterfall.”  So, the reasonable conclusions from this are:

  • Waterfall (or pseudo-waterfall things like RUP) are really out of style.
  • Agile is mainstream, but the real fashionistas are so over it because everyone’s messing it up and ruining it.
  • Cool kids are looking for what’s next: a post-Agile world.

Agile, Distilled

I’m not certified in Scrum or XP or anything else. I have no project management credentials and no letters after my name. I certainly have experience in these methodologies and understand the ceremonies and the roles, but I’d hardly count myself an encyclopedia of how everything should operate. It’s always bemused me that arguments emerge over the finer points of the exact approaches to these methodologies and that people can actually be certified in exact adherence when the focus of the Agile Manifesto seems to me to be best summarized by the idea of bringing pragmatism and empirical experimentation to software development. But, opportunities for snark notwithstanding, it’s easy to see why people become disillusioned. Anytime there is some sort of process and any failures are blamed on a lack of true, absolute adherence, you have a recipe for quackery.

But at it’s core, I think Agile methodologies can be distilled to a single compound principle: tighten the feedback loop and improve the feedback. Think about the different things that are done in various flavors of agile: automated testing/TDD, short sprints, pair programming, retrospectives, big/visible charts, etc. All of it is about delivering feedback faster, delivering feedback more obviously, and acting quickly on the feedback. This is really, really non-controversial. The sooner you get feedback on what you’re working on, the sooner you can improve. So, where does everything go off the rails?

Is it that things don’t go well because people aren’t following the processes right? Is it that the processes themselves are broken? Is it that the processes are good, but a bunch of hucksters are peddling fools’ gold? Is it that people are just too entrenched in their ways to be compatible with these principles? Personally, I don’t think it’s any of this. I think that the fundamental problem is one of expectations (though some of the hucksters do nothing to mitigate this). Specifically, I think the main source of woe is the assumption that “going Agile” will cure what ails you because the only thing standing between the broken, over-budget pile of crap you’re about to ship and software nirvana is having a different set of meetings and adopting a different desk arrangement. The Agile promise isn’t realized by tweaking process; it’s realized by getting a scheme in place for receiving meaningful feedback and using that feedback, with a lot of practice, to get better at making software.

10,000 Hours

Wait, what? Practice? That doesn’t have anything to do with Kanban boards, swim lanes, stand-ups, and all of the other Agile stuff that gets so much focus. In the words of the esteemed Alan Iverson, “What are we talking about? Practice? We’re talking about practice, man.” The wide world is talking about the magic dust you can sprinkle on a team to make it crank out better software, faster, and I’m in here talking about practice.

During my semi-weekly drive to Detroit, I’ve been listening to audio books, including, most recently, Malcom Gladwell’s “Outliers.” Without offering up too much of a spoiler, one of the things that he talks about is that people like Bill Gates don’t reach the incredible levels of success that they do by being dominant in a vacuum, but rather they combine incredible fortune and opportunity with cleverness, ingenuity and a whole lot of really hard work. Bill Gates was born at a time where him being a teenager coincided with the very first machines that allowed for desktop based computing, rather than punch cards. He had the incredible good fortune of gaining access to such a computer in the late 60’s, as a teenager, when almost no adults or professionals on Earth had such access. Addicted to the fun of programming, he worked nights and weekends during his teenage years, programming, at a time when few others in the world had that opportunity. By the time he was an adult, ready to start Microsoft, he was one of the few people on Earth with 10,000 hours of experience programming this way.

The hypothesis for which the book is best known is that 10,000 hours of deliberate practice is the general average amount of time put in before a person acquires mastery of a field. As I understand it, there has been some corroboration of this hypothesis as well as some counter-evidence, but the broad point remains that achieving mastery requires a whole lot of deliberate work and concerted, sustained effort to improve. You know how you get really good at delivering software? Do it for years and years, constantly learning from your mistakes. You know how you don’t get really good at delivering software? By having slightly different meetings, shipping crap more frequently, and hoping for different results.

The way I see it, big A Agile, at its core, is not about fixing you or your software. It’s about making your practice more meaningful and efficient. To return to the basketball metaphor, Agile isn’t a weekend retreat that turns you into Michael Jordan. It’s a weekend retreat that shows you how to do a bunch of layup and dribbling drills that, after 10,000 hours of practice, will make you pretty good, as long as you keep challenging yourself. The great shame of Agile or any other kind of movement is that of false hope in easy fixes. Done well, it will expose your pain points more quickly, provide you with tools for removing obstacles, illuminate a path to improvement, and show you how to self-evaluate regularly. But it won’t make you awesome. Only practice and continuous learning will do that.


Opening Word is Software Development Fail

I’m going to perform a slight experiment on myself in this post. As I type this second sentence, I have only a general idea for a post in mind, and I’m going to go out on a limb and say that this will be a relatively short post. I won’t come back and edit this paragraph if it turns out not to be, however. I’ll just look stupid. I’m going meta here because my posting cadence has slipped and I’m going to aim to combat that by mixing in some relatively short posts that are quicker to write and read. I’ve tried this before, and it’s sort of like a yo-yo diet where I cut back for a bit and then just kind of bloat back out to where I was. Anywho…

Over the course of my career a fairly common thing that I’ve done at a new company is to set up a wiki for collaboration. Almost invariably, this wiki replaces or, at least, aims to replace a series of Word documents. It’s as though there’s some kind of knowledge collection progression that goes, “nothing, README, Word, Wiki (or, perhaps, Sharepoint),” and I make my home at an organization just long enough to say, “hey, there’s a better option than shared drive/source controlled Word documents.” Why is the next step better? Searchability, not needing that “version history” table at the top, sane linking, changing the emphasis to content over styling, etc.


But, I had a thought today that I’d been sort of missing the point all these years. It’s not that wiki isn’t good, by any means, as a collaboration tool, but it’s that I’m often using it to mitigate the symptoms rather than treat the illness. If, as a developer, you find yourself opening Word to document a process, you’ve failed. If you optimize by documenting in a wiki, you’re just failing in a more searchable, sophisticated, and open-source way (unless you use Sharepoint, and then not open-source and maybe not sophisticated… hiyo, just kidding, but kind of not kidding.)

Is this a harsh thing to say? Certainly. Could you argue that I’m link baiting once you hear what comes next? Probably. But I think there’s an element of truth if you allow yourself to de-stigmatize the word “failure” and interpret it without a value judgment. For instance, today, I failed at being someone with 10,000 RSS subscribers to my blog. It’s true, but not damning. Developers documenting things with Word fall into this same category. You’re failing when you do this, and what you’re failing to do is automate.

I’ve blogged about a similar line of thought with code comments in the past. Comments in method bodies are basically developers saying, “I’m going to punt on making this code expressive and just kinda cheat by explaining this mess in English.” So it goes on a broader level with Word documents. Why do you write Word documents, after all? It’s to explain how to setup your development environment or how to perform a deploy. Maybe it’s to document what needs to happen whenever you add a new feature to the code base or in the event that you need to rollback to a previous version of the application. Whatever it is, you’re faced with some kind of procedure and you declare, “there’s a precise sequence of instructions that needs to be executed, and, as a programmer, I’m going to write them in English and use the next poor sap that happens on them as the runtime interpreter.”

I mean, think about it. If your process is defined well enough to merit a series of numbered steps in a Word document, it’s probably defined well enough to automate. Now, it might be that it’d take you three months to automate and 30 minutes to make the Word document. It might be that there are steps you lack the authority or permission to automate (or even do). It might be that you’re making a user manual or API document. There are any number of practical reasons that you’re not some kind of failure as a person for cracking open Word and explaining how to do something with a computer. You’re not a failure, but you have failed. For whatever reason, you’ve failed to automate. So next time you find yourself reflexively starting Word to make some sort of “writeup” about a technical thing, pause and ask yourself, “would automation of this process be possible and worthwhile?” It might not be, but then again, you might be surprised to find that the answer is “yes.”


Imagine a Process Regression

I was recently out to dinner with some friends that are fellow techies, and we were talking about software development methodologies. One of them, who works in a shop that practices pretty true-to-form Scrum, was talking about how management via the product owner was starting to put pressure on them to produce certain, lower point estimates during the planning poker game. He said that it struck the younger team members who’d only worked there as extremely strange and that he personally felt that the powers that be were sort of trying to steer them to a more “waterfall-ish” style. I had a random thought and, rudely, burst out laughing during his anecdote which was, in no way, intended to be funny.

The thing that I had thought of, I explained to my confused friends, was how incredibly weird and counter-intuitive moving to a waterfall development methodology would seem to his teammates who had come into the software development world knowing only Scrum. It’d be like coming home one day and telling your family that you were through buying soap because from now on all you needed was some lye, some elbow grease and the river on the other side of town. Of course, there would (and has been) culture shock going from Waterfall to Agile, but that’s mostly born of a kind of “that’s an impossible pipe dream” mentality. Going the other way has more of a “you want to do what…. and why?” vibe.

Think of this. You’ve never known anything but a disciplined agile approach to software. You have an automated build and deployment setup. Every time you check in code, the build executes, unit tests are run, and the code is pushed to a test environment for exploratory QA and validation of user stories. Checkins are frequent, collaboration is ubiquitous and refactoring and cleaning the code is something you do on a regular basis. Once every other week, you ship the code and it’s no big deal. The (business) day after shipping, you and the team collaboratively plan what you’re going to do for the next two weeks, trusting that someone is prioritizing the work you wind up doing toward a larger technical product/vision. Your users see regular progress and give you regular feedback, creating a tight communication loop and eliminating surprises. And, if the users change their mind about the features they want or don’t like what you’ve done over the last two weeks, you simply spend the next two weeks steering methodically back in the right direction. But don’t worry, that’s all about to change!

Forget about all of that bi-weekly shipping nonsense. We’re going to ship once, as far as we’re concerned. A second shipment is by no means a given because frankly, the first one is rather likely to fail and anger the users. So, let’s not concern ourselves with anything but the first shipment. For the first month or so, we’re going to gather the requirements for the project. We’ll skip through a meadow, plucking them off of the beautiful flora and placing them in our baskets. That’ll be a good month. At the end of that month, we’ll tell our users that they’d better be sure that our endless array of diagrams and thousands of statements about what the software “shall” do matches exactly what they want because once we move onto the next phase, there’s no going back or changing their minds without either expensive change requests or lawyers.

Next up comes the design phase. This is the part where we spend a month figuring out how to write all of the code that we’re going to write over the next several months, but without actually writing that code. While this may sound nuts, the reason that we have to do this is because the labor is going to be divided up in such a way that everyone will kind of go off into their own silo and write code for months prior to an extended “integration phase” where our architect, Dr. Frankenstein, assembles the various individually created pieces into a reasonable attempt at a piece of software and then attempts to breath life into it. In order for that task to be as minimally gruesome as possible, a lot of planning is needed.

The next phase is implementation. Implementation phase is where we start coding and the previous two months of ‘work’ become completely irrelevant. Everyone codes in their silos until integration time. Some attempts are made at early integrations, but those are abandoned when it’s discovered that they are time consuming and the project managers are worried about the fact that they’re already behind schedule. Everyone will then pray that somehow we make up the time that we’ve gotten behind at integration time, but, who are we kidding? That’s not going to happen. The next phase will ideally be the testing phase, but we won’t have time for that. Instead, we’ll start the death march phase, where everyone works 14 hour days, 7 days a week up to the deadline.

Then, my friend, we ship. Tada! Ooh, that wasn’t supposed to happen. Heh, heh, weird. I’m sure the rest of it will be fine if I just — ooh, yikes. What do you mean this screen doesn’t look at all like what you were picturing? Heh, heh, maybe we can reschedule this demo. This is the phase where the thing we chucked over the wall, behind schedule and with a minimum of testing, blows up in our faces. After another month or so of death march following the release, we manage to wrangle the software into a state where, while the source code looks hideous, it’s at least stable and no one is screaming at us anymore.

There, doesn’t that sound better than the one where you’re shipping software every two weeks and not working 90 hour weeks to make up for the fact that you’re not actually very good at predicting the future? But of course, I kid. It was just an interesting exercise for me to think of how the world would seem going from agile to waterfall. Agile methodologies are inherently developer-centric when done correctly. Developers like quick feedback, routine shipments, and process automation because this makes for sustainable, incremental progress. The main complaining I hear from waterfall developers isn’t that agile doesn’t sound good — it’s that it sounds too good to be true. Usually it’s things like “we couldn’t do agile because we’re a shop that does _______” where the blank is some excuse that isn’t actually reasonable unless the company is shipping Mars Rovers. So if you’re already doing agile — you’ve already seen how well it works — going back would seem certifiably crazy. And, you know what? I like that there’s a generation of developers out there that think Waterfall is just a bed time story that the old timers use to scare the newbies into behaving.


Defining Done for Your Deployment Process

A Tale of Debugging

The other day, someone came to me and told me that a component of one of the web applications that my team maintains seemed to have a problem. I sat with her, and she showed me what was going on. Sure enough, I saw the issue too. It was the kind of integration thing for which we were able to muster up some historical record and see approximately when the problem had started. Apparently, the problem first occurred around the last time we pushed a version of the website into production. Ruh roh.


Given that this was a pretty critical issue, I got right down to debugging. Pretty quickly, I found the place in the code where the integration callout was happening, and I stepped through it in the debugger. (As an aside, I realize I’ve often made the case against much debugger use. But when legacy code has no unit, component or integration tests, you really don’t have a lot of options.) No exceptions were thrown, and no obvious problems were occurring. It just hit a third party API, went through it uneventfully, but quietly failed.

At this point, it was time for some tinkering and reverse engineering. When I looked at the method call that seemed to be the meat of the issue, I noticed that it returned an integer that the code I was debugging just ignored. Hovering over it, XML doc comment engine told me that it was returning an error code. I would have preferred an exception, but whatever — this was progress. Unfortunately, that was all the help I got, and there was no indication what any returned error code meant. I ran the code and saw that I was getting a “2,” so presumably there was an error occurring.

Maddeningly, there was no online documentation of this API. The only way I was able to proceed was to install a trial version of this utility locally, on my desktop, and read the CHM file. They offered one through their website, but it was broken. After digging, I found that this error code meant “call failure or license file missing.” Hmm… license file, eh? I started to get that tiny adrenaline rush that you get when a solution seems like it might be just around the corner. I had just replaced the previous deployment process of “copy over the files that are different to the server” with a slightly less icky “wipe the server’s directory and put our stuff there.” It had taken some time to iron out failures and bring all of the dependencies under source control, but I viewed this as antiseptic on a festering sore. And, apparently, I had missed one. Upon diving further into the documentation, I saw that it required some weirdly-named license file with some kind of key in it to be in the web application root’s “bin” folder on the server, or it would just quietly fail. Awesome.

This was confirmed by going back to a historical archive of the site, finding that weird file, putting it into production and observing that the problem was resolved. So time to call it a day, right?

Fixing the Deeper Issue

Well, if you call it a day now, there’s a good chance this will happen again later. After all, the only thing that will prevent this after the next deployment is someone remembering, “oh, yeah, we have to copy over that weird license file thing into that directory from the previous deploy.” I don’t know about you, but I don’t really want important system functionality hinging on “oh, yeah, that thing!”

What about a big, fat comment in the code? Something like “this method call will fail if license file xyz isn’t in abc directory?” Well, in a year when everyone has forgotten this and there’s a new architect in town, that’ll at least save a headache next time this issue occurs. But this is reactionary. It has the advantage of not being purely tribal knowledge, but it doesn’t preemptively solve the problem. Another idea might be to trap error codes and throw an exception with a descriptive message, but this is just another step in making the gap between failure and resolution a shorter one. I think we should try avoid failing at all, though having comments and better error trapping is certainly a good idea in addition to whatever better solution comes next.

What about checking the license file into source control and designing the build to copy it to that directory? Win, right? Well, right — it solves the problem. With the next deploy, the license file will be on the server, and that means this particular issue won’t occur in the future. So now it must be time to call it day, right?

Still no, I’d argue. There’s work to be done, and it’s not easy or quick work. Because what needs to happen now is a move from a “delete the contents of the server directory and unzip the new deliverable” deployment to an automated build and deployment. What also needs to happen is a series of automated acceptance tests in a staging environment and possibly regression tests in a production environment. In this situation, not only are developers reading the code aware of the dependency on that license file, not only do failures happen quickly and obviously, and not only is the license file deployed to where it needs to go, but if anything ever goes wrong, automated notifications will occur immediately and allow the situation to be corrected.

It may seem pretty intense to set all that up, but it’s the responsible thing to do. Along the spectrum of some “deployment maturity model,” tweaking things on the server and forgetting about it is whatever score is “least mature.” What I’m talking about is “most mature.” Will it take some doing to get there and probably some time? Absolutely. Does that mean that anything less can be good enough? Nope. Not in my opinion, anyway.


Easy Deployment: the Alpha and the Omega

A bit of housekeeping…you may have noticed that the social media buttons look a bit different if you’re not accessing through RSS. The old plugin that I was using seems not to be supported anymore, and the Facebook button vanished for a bit. I tried out a replacement and liked it, so I kept it. My thanks to Active Bits for the Social Sharing Toolkit.

Wrong But Fast

There’s a pretty good chance that your deployment process is both too painful and not painful enough. But before I return to that cryptic statement, let me talk a bit about something I’ve observed in developers — especially ones that are newer to the industry. Here’s an example of a series of exchanges that has become pretty familiar to me:

User: It would be nice if the profile screen had a way I could change my password.

Young Buck: That’ll take like, literally, two seconds. I’ll be right back!

(Fifteen minutes later)

Young Buck: Okay, I pushed it out to the server, so you can change your password now.

User: Wow! It’s live already? That’s really cool! Thank you! Let me try it out. Let’s see… oh, hmmm. When I try to log in it looks like it crashes.

Young Buck: That doesn’t seem right. I mean, the only thing I changed was…. oh! I know exactly what happened. Give me three minutes, and I’ll be back.

(30 minutes later)

Young Buck: Alright, should be good.

User: Well, I can get in again, and there’s the change password button, but when I click, nothing happens.

Young Buck: That’s not possible. You must have forgotten to clear your cache. Unless…wait, I think I know what’s going on here. Give me 10 minutes.

User: Uh, tell ya what — just don’t worry about it. Maybe I’ll try it again next week.

Young Buck: (Crestfallen) But it’ll only take 10 minutes and I know exactly what the problem is.

User: (With pity) I’m sure you do, but I’ve got a lot of things to get done today.

This is not professional behavior. Imagine if you took your car in for repairs somewhere and things went this way. You’d probably have the Better Business Bureau on the phone shortly or at least be headed to another mechanic. The young buck is sloppy because he’s brash and arrogant. Right?

Or does it just come off that way a little because of how sure he seems when really he’s just eager to please the user and prove himself? Personally, I find that in the overwhelming majority of cases this is really what’s going on. People often get into programming because they like solving problems. And many programmers were some of the smartest kids in their classes growing up — the ones waving their hands frantically, demanding that the teacher give them a chance to show that they know the answer.


As entry level programmers, the school game has been all that they know. It’s a balance between rushing to get the right answer (teacher calling on students, timed exams, cramming in homework, etc.) and getting answers right, with the former often winning. In college, most programming assignments are evaluated by programs that allow you to submit your code as often as you want and get immediate feedback. There are also office hours, so students who visit professors and TAs the most frequently and submit the most work tend to get the best grades and the most feedback. Computer Science students are used to a paradigm where ideas are valued over execution.

Welcome to the Real World, Grasshopper

In the professional world, however, execution reigns supreme. Ideas are cheap. You may be able to rattle off the quick-sort algorithm in pseudo-code faster than anyone around you, but that’s not going to win you any startup capital. With software, even the intellectual property system (USPTO, anyway) is a joke seemingly designed to let Apple, Microsoft and Google endlessly sue each other and occasionally to swat little guys like bugs. Having the idea first and/or quickly is not as important as getting the idea right in the end.

It takes people some time to learn this upon entering the work force, and exchanges like the one I mention above are common. Users more or less say, “Go away and come back to me when you have something that makes my life better and not a second before. I don’t care if you thought of it in five seconds or if it took you five minutes or if it would have taken you five minutes except that the database was actually not normalized to BCNF and blah, blah, blah. Whatever.” Some people figure this out quickly, and some never figure it out. But whatever the speed may be and however much your group may or may not have come to terms with this, there needs to be structure in place to stop the madness.

In other words, a lot of the developers in your group are going to be eager to please. This is especially true if they regularly interact with their users. There is going to be pressure on them to say things like, “sure, I’ll have that for you in 10 minutes.” But they need not to say things like that. If they can say things like that and they can successfully (attempt to) make them happen, your deployment process is not painful enough.

Hurts so Good

Deployment is not to be taken lightly, especially if there is a release and the users are going to be seeing the result of the work. If you can deploy effortlessly in minutes, there’s a very good chance your process is not painful enough. The situation I’ve been describing above suffers from this very problem — it’s too easy for eager crowd-pleasers to deploy and thus it’s too easy for them to depend on users to be their fast feedback mechanism.

Developers are smart and often opportunistic. Getting fast feedback is extremely important to them, and they’ll naturally seek out ways to procure it. If you let them, they’ll use end users as their feedback mechanism (and, in a tone-deaf sense that ignores end-user perception, this is actually optimal), but you can’t permit this. Rather than following that path of least resistance, or at least familiarity from school/hobbyist days, you need to choke off that path and force them to carve a new riverbed. You need to make deployment more painful.

Now there’s “antiseptic on a cut” painful and “shark gnawing on your leg” painful. You want to gun for the former. A lot of deployment processes that enable developers to SPAM end users with non-functional updates are the product of amateur hour: xcopying files to the server, putting an executable on a share drive, zipping things up and emailing them, etc. These things tend to be both easy to do and easy to botch, so simply setting policies in place that prevent developers from doing them is antiseptic on the cut.

Deployments and especially releases to end users need to follow some process, and coding is simply one stop along the way — not the entirety of it. Ideally, there should be the coding and then developer testing, but from there, automated unit and integration tests, code reviews, static analysis, exploratory/manual testing by QA and observation by a UX group can all be part of the mix. These good practices serve to improve quality in and of themselves, but they also serve to prevent spurious, sloppy releases. If you know you can make a change in five minutes and have it in front of a user in six minutes, you’re going to do that. But if you know that you can make the change in five minutes and it’ll be days of going through the entire release process before you hear back from the end user, you’ll start finding other ways to get fast feedback (such as running unit tests, asking other developers to take a look and working closely with QA). By making deployment more painful you ensure that a lot more care goes into it.

But Hurts Too Much

If you’ve been reading along and grinding your teeth in objection to my premise that your deployment process needs to hurt more, I understand. Things shouldn’t be painful, and deliberately hurting yourself is a form of madness. If your deployment process hurts, it’s too painful, even though I just told you it’s not painful enough. But you have to prevent pain in the right way. Xcopy deployment is like being a boxer addicted to morphine — your process is horrible but pain free. Now imagine that you realize that being addicted to drugs is a problem, so you cut back and start feeling the pain each time a professional puncher unloads on you. In one sense, you’re feeling too much pain because it hurts to get punched in the face, but at the same time you’re not feeling enough pain because the amount you’re feeling isn’t causing you to consider another vocation.

The analogy here may not line up exactly, but the idea is similar. A lot of development and deployment processes are problematically painful in that they’re error-prone and difficult, but not painful enough in that they don’t prevent over-eager deploys and bad decisions. The solution? Get off the junk and stop letting yourself get pummeled by human wrecking balls. Or, in software terms, have a process that makes bad deployments hard and good ones very easy.

Now, getting to this release nirvana is not, itself, simple, but life is simple once you get there. The path to it generally involves a lot of automated testing and good planning. It involves a predictable release cadence, such as a sprint, and a commitment to following the process, not making exceptions nor cramming things in at the 11th hour or pushing back release dates. It involves continuous integration rather than periodic, nightmarish feature branch integrations. It involves a resistance to patching frantically when you make mistakes (you’ll learn a valuable lesson for next time). It involves a simple, fully-automated, easily repeatable build process. It involves a single click/button push deployment process. Summed up, it means that every time someone on your team checks in code, a series of automated tests and checks ensure that the code would be a good candidate for production or it is rejected from checkin until it would be. It means that your code could be shipped with a reasonably high degree of confidence on every checkin and that whether or not to actually ship is a business decision — not a technical one.

I encourage you to take a look at your build process and deployment process. Is it easy because Jim in accounting could do it? If so, it’s too easy and it’s definitely causing you problems. Is it hard enough that you do a lot of checking beforehand because you won’t want to do it again if things go wrong? That’s an improvement because it forces your hand for early testing and vetting, but it’s still a time-wasting problem. First, think about putting obstacles in place to guard against careless deployment, then think about refining those obstacles into process-helping practices, and finally, think about smoothing over the obstacles in the form of complete automation, leaving nothing but good, easy process.


It’s a Work in Progress

I’ll have that for you next week. Oh, you want it to do that other thing that we talked about earlier where it hides those controls depending on what the user selects? I’ll have it for you next month. Oh, and you also want the new skinning and the performance improvements too? I’ll get started and we’ll set about six months out as the target date. Unless you want to migrate over to using Postgres. Then make it a year.

Have you ever had any conversations that went like this? Ever been the person making this promise or at least the one that would ultimately be responsible for delivering it? If so, I bet you felt like, “man, I’m totally just making things up, but I guess we’ll worry about that later.” Have you ever been on the hearing end of this kind of thing? I bet you felt like, “man, he’s totally just making things up, and none of this will ever actually happen.”

As you’ll know if you’ve been checking in at this blog for a while, my opinions about the “waterfall methodology” are no secret. It isn’t my intention to harp on that here but rather to point out that the so-called waterfall approach to software development is simply part of what I consider to be a larger fail: namely, putting a lot of distance between promises and deliveries.

I get the desire to do this–I really do. It’s the same reason that we root for underdogs in sports or we want the nice guy not to finish last. Having success is good, but sudden, unexpected success is awesome. It creates a narrative that is far more compelling than “guy who always delivers satisfactory results does it yet again.” Instead, it says, “I just pulled this thing called an iPod out of nowhere and now I’m going to be the subject of posthumous books and cheesy project manager motivational calendars in a decade.”

Companies have done things like this for a long, long time–particularly companies that sell tangible goods. It’s been the story of a consumer-based world where you innovate, patent, and then sell temporarily monopolized widgets, making a fortune by giving the world something new and bold. You probably wind up on the cover of various magazines with feel-good stories about the guy behind the revolutionary thing no one saw coming, except for one visionary (or a team of them) toiling away in anonymity and secrecy, ready to pull back the curtain and dazzle us with The Prestige.


But here’s the thing. The world is getting increasingly service oriented. You don’t really buy software anymore because now you subscribe and download it from “the cloud” or something. Increasingly, you rent tools and other things that perhaps you would previously have purchased. Even hardware goods and electronics have become relatively disposable, and there is the expectation of constant re-selling. Even the much-ballyhooed innovator, Apple, hasn’t really done anything interesting for a while. We’re in a world where value is delivered constantly and on a slight incline, rather than in sudden, massive eruptions and subsequent periods of resting on laurels.

What does the shifting global economic model have to do with the “waterfall” method of getting things done? Well, it pokes a hole in the pipe-dream of “we’ll go off and bury ourselves in a bunker for six months and then emerge with the software that does everything you want and also makes cold fusion a reality.” You won’t actually do that because no one really does anymore, and even back when people did, they failed a lot more often than they succeeded in this approach.

I love to hear people say, “it’s a work in progress.” That means that work is getting done and those doing it aren’t satisfied, and those two components are essential to the service model. These words mean that the speaker is constantly adding value. But more than that, it means that the speaker is setting small, attainable goals and that those listening can expect improvements soon and with a high degree of confidence. The person doing the work and the target audience are not going to get it perfect this week, next week, next month, or maybe ever. But they will get it better and better at each of those intervals, making everyone involved more and more satisfied. And having satisfaction steadily improve sure beats having no satisfaction at all for a long time and then a very large chance of anger and lawsuits. Make your work a work in progress, and make people’s stock in your labor a blue chipper rather than a junk bond.


What Drives Waterfall Projects?

To start off the week, I have a satirical post about projects developed using the waterfall ‘methodology.’ (To understand the quotes, please see my post on why I don’t think waterfall is actually a methodology at all). I figured that since groups that use agile approaches and industry best practices have a whole set of xDD acronyms, such as TDD, BDD, and DDD, waterfall deserved a few of its own. So keep in mind that while this post is intended to be funny, I think there is a bit of relevant commentary to it.

Steinbeck-Driven Development (SDD)

For those of you who’ve never had the pleasure to read John Steinbeck’s “Of Mice and Men,” any SDD practitioner will tell you that it’s a heartwarming tale of two friends who overcome all odds during the Great Depression, making it cross-country to California to start a rabbit petting zoo. And it’s that outlook on life that they bring to the team when it comes to setting deadlines, tracking milestones, and general planning.


Relentlessly optimistic, the SDD project manager reacts to a missed milestone by reporting to his superiors that everything is a-OK because the team will just make it up by the time they hit the next one. When the next milestone is missed by an even wider margin, same logic applies. Like a shopping addict or degenerate gambler blithely saying, “you gotta spend money to make money,” this project manager will continue to assume on-time delivery right up until the final deadline passes with no end in site. When that happens, it’s no big deal–they just need a week to tie up a few loose ends. When that week is up, it’ll just be one more week to tie up a few loose ends. When that week expires, they face reality. No, just kidding. It’ll just be one more week to tie up a few loose ends. After enough time goes by, members of the team humor him with indulgent baby talk when he says this: “sure it will, man, sure it will. In a week, everything will be great, this will all be behind us, and we’ll celebrate with steaks and lobster at the finest restaurant in town.”

Spoiler alert. At the end of Steinbeck’s novel, the idyllic rabbit farm exists only in the mind of one of the friends, shortly before he’s shot in the back of the head by the other, in an act that is part merciful euthanasia and part self-preservation. The corporate equivalent of this is what eventually happens to our project manager. Every week he insists that everything will be fine and that they’re pretty close to the promised land until someone puts the project out of its misery.

Shooting-Star-Driven Development (SSDD)

Steinbeck-Driven Development is not for everyone. It requires a healthy ability to live in deluded fantasy land (or, in the case of the novel, to be a half-wit). SSDD project managers are not the relentless optimists that their SDD counterparts are. In fact, they’re often pretty maudlin, having arrived at a PM post on a project that everyone knows is headed for failure and basically running out the clock until company bankruptcy or retirement or termination or something. These are the equivalents of gamblers that have exhausted their money and credit and are playing at the penny tables in the hopes that their last few bucks will take them on an unprecedented win streak. Or, perhaps more aptly, they’re like a lonely old toy-maker, sitting in his woodshop, hoping for a toy to come to life and keep them company.

This PM and his project are doomed to failure, so he rarely bothers with status meetings, creates a bare minimum of power points, and rarely ever talks about milestones. Even his Gantt charts have a maximum of three nested dependencies. It’s clear to all that he’s phoning it in. He knows it’s unlikely, but he pins his slim hope to a shooting star: maybe one of his developers will turn out to be the mythical 100x developer that single-handedly writes the customer information portal in the amount of time that someone, while struggling to keep a straight face, estimated it would take to do.

As projects go along and fall further and further behind schedule and the odds of a shooting star developer become more and more remote, the SSDD project manager increasingly withdraws. Eventually, he just kind of fades away. If Geppetto were a real life guy, carving puppets and asking stars to make them real children, he’d likely have punched out in an 19th century sanitarium. There are no happy endings on SSDD projects–just lifeless, wooden developers and missed deadlines.

Fear-Driven Development (FDD)

There is no great mystery to FDD projects. The fate of the business is in your hands, developers. Sorry if that’s a lot of pressure, but really, it’s in your hands.

The most important part of a FDD project is to make it clear that there will be consequences–dire consequences–to the business if the software isn’t delivered by such and such date. And, of course, dire consequences for the business are pretty darned likely to affect the software group. So, now that everyone knows what’s at stake, it’s time to go out and get that complex, multi-tiered, poorly-defined application built in the next month. Or else.

Unlike most waterfall projects, FDD enters the death march phase pretty much right from the start of coding. (Other waterfall projects typically only start the death march phase once the testing phase is cancelled and the inevitability of missing the deadline is clear.) The developers immediately enter a primal state of working fourteen hours per day because their very livelihoods hang in the balance. And, of course, fear definitely has the effect of getting them to work faster and harder than they otherwise would, but it also has the side effect of making the quality lower. Depending on the nature of the FDD project and the tolerance level of the customers for shoddy or non-functional software, this may be acceptable. But if it isn’t, time for more fear. Consequences become more dire, days become longer, and weekends are dedicated to the cause.

The weak have nervous breakdowns and quit, so only the strong survive to quit after the project ends.

Passive-Aggressive-Driven Development (PADD)

One of the most fun parts of waterfall development is the the estimation from ignorance that takes place during either the requirements or design days. This is where someone looks at a series of Visio diagrams and says, “I think this project will take 17,388.12 man-hours in the low risk scenario and 18,221.48 in the high-risk scenario.” The reason I describe this as fun is because it’s sort of like that game you play where everyone guesses the number of gumballs in a giant jar of gumballs and whoever is closest without going over wins a prize. For anything that’s liable to take longer than a week, estimation in a waterfall context is a ludicrous activity that basically amounts to making things up and trying to keep a straight face as you convince yourself and others that you did something besides picking a random number.

Well, I broke this task up into 3,422 tasks and estimated each of those, so if they each take four hours, and everything goes smoothly when we try to put them all together with an estimate for integration of… ha! Just kidding! My guess is 10,528 hours–ten because I was thinking that it’d have to be five digits, the fve because it’s been that many days that we’ve been looking at these Gantt charts and sequence diagrams, and twenty-eight because that was my number in junior high football. And you can’t bid one hour over me because I’m last to guess!

But PADD PMs suck all of the fun out of this style of estimation by pressuring the hours guessers (software developers) into retracting and claiming less time. But they don’t do it by showing anger–the aggression is indirect. When the developer says that task 1,024, writing the batch file import routine, will take approximately five hours, the PADD PM says, “Oh, wow. Must be pretty complicated. Jeez, I just assumed that a senior level developer could bang that out in no more than two. My bad.” Shamed, the developer retracts: “No, no–you’re right. I figured the EDI would be more complicated than it was, so I just realized that my estimate is actually two hours.”

Repeated in aggregate, the PADD PM is some kind of spectacular black belt/level 20/guru/whatever metric is used to measure PM productivity, because he just reduced the time to market by 60% before a single line of code was ever written. Amazing! Of course, talk at the beginning of the project is cheap. The real measure of waterfall project success is figuring out who to blame and getting others to absorb the cost when the project gets way behind schedule. And this is where the PADD master really shines.

To his bosses, he says, “man, I guess I just had too much faith in our guys–I mean, I know you hire the best.” To the developers, he says, “boy, your estimates seemed pretty reasonable to me, so I would have assumed that everything would be going on time if you were just putting in the hours and elbow grease… weird.” To the end-users/stakeholders, he says, “it’s strange, all of our other stakeholders who get us all of the requirements clearly and on time get their software on time–I wonder what happened here.”

There’s plenty of blame to go around, and PADD PMs make sure everyone partakes equally and is equally dissatisfied with the project.

Acknowledgements | Contact | About | Social Media