DaedTech

Stories about Software

By

Why Production Monitoring Can Come Too Late

Editorial Note: I originally wrote this post for the Stackify blog.  You can check out the original here, at their site.  While you’re there, have a look around at how their offering can help you hunt down issues from development to production.

I’ve spent a number of years, now, writing software.  At the risk of dating myself, I worked on software in the early 2000s.  Back then, you couldn’t take quite as much for granted.  For example, while organizations considered source control a good practice, forgoing it wouldn’t have constituted lunacy the way it does today.

As a result of the different in standards, my life shipping software looked different back then.  Only avant garde organizations adopted agile methodologies, so software releases happened on the order of months or years.  We thus reasoned about the life of software in discrete phases.  But I’m not talking about the regimented phases of the so-called “waterfall” methodology.  Rather, I generalize it to these phases: build, prep, run.

During build, you mainly solved the problem of cranking through the requirements as quickly as possible.  Next up, during prep, you took this gigantic sprawl of code that only worked on dev machines, and started to package it into some kind of deployable product.  This might have meant early web servers or even CDs at the time.  And, finally, came run.  During run phase, you’d maintain vigilance, waiting for customer issues to come streaming in.

Bear in mind that we would, of course, work to minimize bugs and issues during all of these phases.  But at that time with most organizations, having issues during the “run phase” constituted a good problem to have.  After all, it meant you had reached the run phase.  A shocking amount of software never made it that far.

Monitoring and Software Maturity

We’ve come a long way.  As I alluded to earlier, you’d get some pretty incredulous looks these days for not using source control.  And you would likewise receive incredulous looks for a release cycle spanning years, divided into completely disjoint phases.  Relatively few shops view their applications’ production behavior as a hypothetical problem for a far-off date anymore.

We’ve arrived at this point via some gradual, hard-won victories over the years.  These have addressed the phases I mentioned and merged them together.  Organizations have increasingly tightened the feedback loop with the adoption of agile methodologies.  Alongside that, vastly improved build and deployment tooling has transformed “the build” from “that thing we do for weeks at the end” to “that thing that happens with every commit.”  And, of course, we’ve gotten much, much better at supporting software in production.

Back in the days of shrink-wrap software and shipping CDs, users reported problems via phone call.  For a solution, they developed workarounds and waited for a patch CD in the mail.  These days, always-connected devices allow for patches with arbitrary quickness.  And we have software that gets out in front of production issues, often finding them even before users do.

Specifically, we now have sophisticated production monitoring software.  In some cases, this means simply watching for outages and supplying alerts.  But we also have sophisticated application performance monitoring (APM) capabilities.  As I said, we’ve come a long way.

Read More

By

Transitioning from Manual to Automated Code Review

Editorial note: I originally wrote this post for the SubMain blog.  You can check out the original here, at their site.  While you’re there, have a look at CodeIt.Right.

I can almost sense the indignation from some of you.  You read the title and then began to seethe a little.  Then you clicked the link to see what kind sophistry awaited you.  “There is no substitute for peer review.”

Relax.  I agree with you.  In fact, I think that any robust review process should include a healthy amount of human and automated review.  And, of course, you also need your test pyramid, integration and deployment strategies, and the whole nine yards.  Having a truly mature software shop takes a great deal of work and involves standing on the shoulders of giants.  So, please, give me a little latitude with the premise of the post.

Today I want to talk about how one could replace manual code review with automated code review only, should the need arise.

Why Would The Need for This Arise?

You might struggle to imagine why this would ever prove necessary.  Those of you with many years logged in the enterprise in particular probably find this puzzling.  But you might find manual code inspection axed from your process for any number of reasons other than, “we’ve decided we don’t value the activity.”

First and most egregiously, a team’s manager might come along with an eye toward cost savings.  “I need you to spend less time reading code and more time writing it!”  In that case, you’ll need to move away from the practice, and going toward automation beats abandoning it altogether.  Of course, if that happens, I also recommend dusting off your resume.  In the first place, you have a penny-wise, pound-foolish manager.  And, secondly, management shouldn’t micromanage you at this level.  Figuring out how to deliver good software should be your responsibility.

But let’s consider less unfortunate situations.  Perhaps you currently work on a team of 2, and number 2 just handed in her two week’s notice.  Even if your organization back-fills your erstwhile teammate, you have some time before the newbie can meaningfully review your code.  Or, perhaps you work for a larger team, but everyone gradually becomes so busy and fragmented in responsibility as not to have the time for much manual peer review.

In my travels, this last case actually happens pretty frequently.  And then you have to chose: abandon the practice altogether, or move toward an automated version.  Pretty easy choice, if you ask me.

Read More

By

Things Everyone Forgets Before Committing Code

Editorial note: I originally wrote this post for the NDepend blog.  You can check out the original here, at their site.  While you’re there, take a look around at some of the other posts as well.

Committing code involves, in a dramatic sense, two universes colliding.  Firstly, you have the universe of your own work and metaphorical workbench.  You’ve worked for some amount of time on your code, hopefully in a state of flow.  And secondly, you have the universe of the team’s communal work product.  And so when you commit, you force these universes together by foisting your recent work on the team.

In bygone years, this created far more heartburn for the average team than it does today.  Barbaric as it may seem, I can actually remember a time when some professional software developers didn’t use source control.  A “commit” thus involved literally overwriting a file on a shared drive, obliterating all trace of the previous version.  (Sometimes, you might create a backup copy of the folder).  Here, your universe actually kind of ate the team’s communal universe.

More Frequent Commits, Fewer Problems

But, even in the earliest days of my career, lack of source control represented sloppy process.  I remember installing the practice in situations that lacked it.  But even with source control in place, people tended to go off and code in their own world for weeks or even months during feature development.  Only when release time neared did they start to have what the industry affectionately calls “merge parties,” wherein the team would spend days or weeks sorting out all of the instances where their changes trampled one another’s.

In the interceding years, the industry has learned the wisdom of continuous integration (CI).  CI builds on the premise, “if it hurts, do it more,” by encouraging frequent, lower stakes commits.  These days, most teams commit on the order of hours, rather than weeks or months.  This significantly lowers the onerousness of universes colliding.

But it doesn’t eliminate the problem altogether, even in teams that live the CI dream.  No matter how frequently you do it and how sophisticated the workflows around modern source control, you still have the basic problem of putting your stuff into the team’s universe.  And this comes with the metaphorical risk of leaving your tools laying around where someone can trip over them.

So today, let’s take a look at some of the most common things everyone forgets before committing code.  And, for the purposes of the post, I’ll remain source control agnostic, with the parlance “commit” meaning generally to sync your files with the team’s.

Read More

By

Pulling Your Team Through a Project Crunch

Editorial note: I originally wrote this post for the NDepend blog.  You can check out the original here, at their site.  While you’re there, download NDepend and take it for a test drive.

Society dictates, for the most part, that childhood serves as a dress rehearsal for adulthood.  Sure, we go to school and learn to read, write, and ‘rithmetic, but we also learn life lessons.  And these lessons come during a time when we can learn mostly consequence-free.

During these formative years, pretty much all of us learn about procrastination.  More specifically, we learn that procrastination feels great.  But then, perhaps a week later, we learn that procrastination actually feels awful.  Our young brains learn a lesson about tradeoffs.  Despair.com captures this with a delightfully cynical aphorism: “hard work often pays off after time, but laziness always pays off now.”

The fact that we laugh at this indicates something interesting.  Whereas we learn most childhood lessons and internalize them (e.g. don’t touch a hot stove), we don’t learn all of them this way.  Procrastination falls into this latter category.  We learn its perils, but we do it anyway, for various reasons.

As someone in the software industry, I suspect you can appreciate this.  Even if you diligently get out in front of all of your work, your organization may not follow suit.  So I’m sure you’ve seen your fair share of so-called “crunch time” over the years.  Crunch time occurs when you put a disproportionate amount of effort right before an important milestone.  Kinda like waiting until the night before the due date to write a 10 page essay on To Kill a Mockingbird.

Today, I’d like to talk about surviving these crunches.  How can you navigate them to come out on the other end with your team’s morale intact?

Beware the Death March

For argument’s sake, I’ll draw a fine line.  Crunch time and the so-called “death march” can seem like the same thing, but I submit that they differ importantly.  Crunch time involves working hard in a spurt to succeed.  Death marches involve working hard indefinitely with a high probability of failing anyway.

The first step for pulling a team through a crunch is to ensure that you’re not instead dispatching them on a death march.  If your organization has the unfortunate habit of getting itself into death marches, you have serious problems on your hands that transcend the immediate future.  At an organization like this, I’d recommend looking for escape avenues and hinting to others that they might do the same.

If this sounds bad or like the height of disloyalty, I suggest a bit of perspective.  Your organization is about to ask you to burn yourself out on an indefinite timeline for something that will probably fail.  Unless you’re a masochist, what about that warrants your loyalty?

Read More

By

Choosing an Acceptance Test Framework

Editorial note: I originally wrote this post for the Monitis blog.  You can check out the original here, at their site.  While you’re there, take a look at their monitoring solutions.

I can still remember writing my first automated tests in a professional setting.  Notice that I didn’t say unit tests, and for good reason.  At the time, some 14 years ago, I hadn’t heard of unit tests.  Instead, I simply automated the process of manually testing my software.

This may sound somewhat facile, but it actually speaks to core principles in the programming profession.  As a relatively inexperienced programmer, I understood the importance of testing my work.  I also understood the importance of automating manual, error-prone process.  And so, of my own accord, I examined and then automated my previously manual testing efforts.

Don’t get me wrong.  By doing this, I reinvented a wheel simply because I did not know of its existence.  Folks had created automated unit test frameworks for this exact purpose.  Had I known, I could have better spent my time learning and using these things.  But, in spite of the waste, I did learn something.  I learned that, under the covers, test frameworks just represented yet another instance of automating an important manual process.

What is User Acceptance Testing (UAT)?

I led with a tale about unit tests because testing software components applies to everyone that writes software.  I mean, you always test your own software, even if you don’t think you’re doing so.  Whenever you compile, you test your code to see if it compiles.  Granted, you aren’t executing the most sophisticated, high-value test known to man.  But you are performing a test of sorts.

But what if we look beyond our own dev boxes a bit?  What if we look at other forms of testing?

For almost any software that we write, other stakeholders will perform other sorts of tests.  These stakeholders includes users or user-proxies, who perform an activity known as user acceptance testing (UAT).  In its simplest incarnation, this involves users or their proxies using the software and evaluating whether or not they find it acceptable.

This can come in various shapes and sizes.  In some cases, actual users perform formalized beta tests, perhaps for pay.  In other cases, someone from the QA group might do a quick run-through and give a thumbs-up or thumbs-down.  But whatever happens, these tests capture the user’s experience and perspective.

Read More