Stories about Software


Years of Lessons Learned from Home Automation

I’ve had three variations of my home automation setup. The first incarnation was a series of Linux command line utilities and cron jobs. There was some vague intention of a GUI, but that never really materialized. The second was a very enterprise-y J2EE implementation that involved Spring, MongoDB, layered architecture, and general wrangling with java. The current and most recent reboot involves nodes in a nod to Service Oriented Architecture and the “Internet of Things.” As I mentioned in my last post, the I’m turning a Raspberry Pi into a home automation controlling REST endpoint and then letting access and interaction happen in more distributed, ad-hoc fashion.

The flow of these applications seems to reflect the trajectory of my career from entry level developer to architect — from novice and hobbyist to experienced professional and, well, still hobbyist. And I think that, to an extent, they also reflect the times and trends in technology. It’s interesting to reflect on it.

When I started out as a programmer in the working world, I was doing a lot in the Linux world with C and C++. In that world, there was no cred to writing any kind of GUI — it was all about being close to the metal, and making things work behind the scenes. GUIs were for the faint of heart. I wrote drivers and kernel space code and automated various interactions between hardware and software. This mentality was carried over into the world of hobby when I discovered home automation. X10 was the province of hobbyist electrical engineers who wrote code out of necessity, and I fell in nicely with this approach. It was all about banging away, hacking, and making things work. Architecture, planning, testing, deployment strategies, etc… who cares? Making it work was all that mattered. I was a beginner.

As my career wound on, I started doing more and different kinds of programming. I found my way into web development with Java, did things in the .NET space, worked with databases, and started to become interested in architecture, software processes and honing my craft. With my newfound knowledge of a breadth of technologies and better software development approaches, I decided on a home automation reboot. I chose Linux and Java to keep the budget as shoe-string as possible. For a server, I could use the machine I took with me to college — a 400 MHz P2 processor and 384 meg of RAM. The hardware, OS, and software were thus all free, and all I had to do was pop for the X10 modules at $10-$20 a piece. Not too shabby.

I was cost conscious, and I had a technical vision for the architecture. I knew that if I created a web application on the server that what I did would be accessible from anywhere: Windows computers, Linux computers, even cell phones (which were a lot more limited as nodes in a network 5-6 years ago when I started laying this out). Java was a good choice because it gave me a framework to integrate all of the different functionality that I could imagine. And I imagined plenty of it.

There was no shortage of gold plating. Part of this was because I was interested in learning new technologies as long as I was doing hobby work and part of this was because I hadn’t yet learned the value of limiting myself to the minimum set of features needed to get going. I had advanced technically enough to see the value in architecture and having a plan for how I’d handle future added features, but I hadn’t advanced enough to keep the system flexible without putting more in up front than I needed. A web page with a link for turning a lamp on may not need data access, domain, service, and presentation layers. And, while I had grand plans to integrate things like home inventory management, recipe tracking, a family calendar and more, those never actually materialized due to how busy I tend to be. But I was practicing my craft and teaching myself these concepts by exploring them, so I don’t look back ruefully. Lesson learned.

Now, I’m rebooting. My old P2 machine is dying slowly but surely, and I recently purchased a lake house where I want to replicate my setup. I don’t have another ancient machine, and it’s time to get more repeatable anyway. A minimal REST endpoint on a Raspberry Pi is cheap and repeatable, and it lets me build the system in my house(s) more incrementally and flexibly. If I want to use WPF to build a desktop app for controlling the thing, then great. If I want to use PHP or Java on a server, then also great. ASP MVC, whatever. Anything that can speak REST will work, and everything speaks REST.

Maybe in another three years, I’ll do the fourth reboot and think of how silly I was “back then” in 2013. But for now, I’ll take the lessons that I’ve learned in my reboots and reflect. I’ve learned that software is about solving problems for people, not just for the sake of solving the problem. A cron job that I can tweak turns my lights on and off, but it doesn’t present any way that the system isn’t weird and confusing for my non-technical girlfriend. I’ve learned that building more than what you need right now is a guarantee that you’ll have more complexity than you need and less benefit. I’ve learned that a system composed of isolated, modular components is better than a monolithic juggernaut that can handle everything. And, most importantly, I’ve learned that you’ve never really got it all figured out; whatever grand plan you have right now is going to need constant care and refinement.


RESTful Home Automation

Here are the general steps to get a REST service going on your Raspberry Pi, using Python. One thing that I’ve learned from blogging over the last several years is that extremely detailed, granular how-tos tend to be the most yawned-at posts. So this is just a quick overview of how you can accomplish the goal without going into a lot of detail. If you want that detail, you can drill into the links I’m providing or else feel free to ask questions in comments or via email/twitter.

  1. Go out and buy these things: USB transceiver, plugin transceiver, lamp module (optional)
  2. On your Pi, install apache with sudo apt-get install apache2.  This is the web server.
  3. Also on your Pi, install web2py with sudo apt-get install python-webpy.  This is the module that makes setting up a REST service a snap.
  4. Install some driver dependencies (I will probably later roll these into what I’m doing) with “sudo apt-get install libusb-1.0 python-usb”.  Here are more detailed instructions from the page of the home automation python driver that I’m using.
  5. Follow the instructions on that page for disabling interfering kernel drivers.
  6. Give your user space account permissions to hit the USB device from the referenced instruction page, but note a typo where he says “sudo nano /etc/udevrules.d/cm19a.rules” and you really want
    “sudo nano /etc/udev/rules.d/cm19a.rules”.
  7. Now go get my stuff from github and run a web server using python rest.py <port>

That’s all there is to it.  Right now, at the time of writing, you would go to http://<your pi’s ip>:<port>/office/on to basically turn everything from A on.  A and office are both hard-coded, but that’s going to change in the next few days as I grow this service to support adding rooms and lights via PUT, and storing them as JSON documents on the server.  You’ll be able to add a light with PUT, supplying the room, light, and X10 code, and then you’ll subsequently be able to toggle it with http://pi:port/room/light/{on/off}

You can also just install Andrew’s driver and use it as-is.  It even has a web mode that supports query parameters.  The reason I didn’t do things that way is because (1) I wanted a REST service and (2) I wanted to be able to customize everything as I went while learning a new language.


The Hidden Cost of Micromanagement

I think everyone has encountered the micromanaging type at some point or another in their career. Most obviously or memorably, this might have been a boss who took the attitude, “no decision too small!” But it might also have been a coworker’s boss or just someone else at the company you had the misfortune to share an office with. If you’ve managed to avoid this your whole career, count yourself fortunate, but understand that there is a type of person out there that feels as though he is the only one capable of making decisions and then acts accordingly.

The up-front cost of this behavior is fairly obvious. It clearly doesn’t scale well, so micromanagers wind up being huge bottlenecks to productivity with all decisions essentially on hold until that person gets to them. Micromanaging also the effect of creating learned helplessness in those around them, rendering everyone else less productive. People learn that there’s no point in trying to have autonomy, so they check out and stop paying attention. Some will even engage in passive aggressive behaviors that harm the organization, such as malicious compliance. “You want me to ship this code as-is, huh? Okie-dokie.” (“Heh, heh, well, it crashes, but hey, you’re the boss!”)

Being subjected to a micromanager is hard to take and often stressful, so turnover is typically high in such scenarios. And that’s where a hidden cost comes in. People working under the purview of a micromanager don’t suddenly and automatically switch gears when their situation improves — they often bring their previous behaviors along for the ride. If you’ve ever seen the movie The Shawshank Redemption, recall the scene with Red in the grocery store when he gets paroled to work after 40 years in prison:

Red: Restroom break?
Boss: You don´t need to ask me every time you need to go take a piss. Just go.
Red (thinking to himself): Forty years I´ve been asking permission to piss. I can´t squeeze a drop without say-so.

Red’s boss and the grocery store are experiencing what I’m describing as the hidden cost of micromanagement. He has an employee so used to having his day micromanaged that the employee is maddeningly, annoyingly dependent. In your own travels, it’s easy to tell the signs of a previous victim of micromanagement: endless updates about details and inconsequential minutiae, constantly asking permission to do anything, non-stop preemptive apologies and general insecurity about work quality. That all might seem harmless, if a little sad, but the problem is that this lack of autonomy and willingness to take charge of situations has a real productivity hit for both parties. The new manager/lead should be focusing on bigger picture issues, such as removing impediments from the path of her team — she shouldn’t have to worry about whether one of her team members’ Outlook inbox is 80% full. That’s a meaningless detail that nobody should report to anyone, and yet tons and tons of micromanagement victims do.

It goes without saying that you should avoid micromanagement. It’s a terrible interaction and leadership strategy on its face that is largely the purview of people with psychological problems. But look for past victims of micromanagement and given them a hand or a leg up. Encourage them. Work with them. And most of all, let them know that they’re capable of making good decisions and worthy of being trusted to do so. Your organization will be much better off for it.


Lessons in Good Naming through Absurdity

There’s something I’ve found myself stressing a lot lately to people on my team. It’s a thing that probably seems to most like nitpicking, but I think is one of the most, if not the most, under-stressed aspects of programming. I’m talking about picking good names for things.

I’ve seen that people often give methods, variables and classes abbreviated names. This has roots in times where saving characters actually mattered. Methods in C had names like strcat (concatenate a string), and that seems to have carried forward to modern languages with modern resource situations. The reader of the method is left to try to piece together what the abbreviation means, like the recipient of a text message from someone who thinks that teenager text-speak is cute.

There are other naming issues that occur as well. Ambiguity is a common one, where methods have names like “OnClick” or even “DoStuff.” You’ll also have methods that are occasionally misleading — a method called “ReadRecords” that reads in some records and then actually updates them as well. Giving this a simple name like “ReadAndUpdateRecords” would take care of this, but people don’t do it. There are other examples as well.

All of this probably seems like nitpicking, as I said a moment ago, but I contend that it isn’t. Code is read way, way more often than it is written/modified, and it’s usually read by people who don’t really understand what was going through the coder’s mind at the time of writing. (This can even include the original coder, revisiting his own code weeks or months later.) Anything that furthers understanding becomes important for saving minutes or even hours spent trying to understand. Methods with names that accurately advertise what they do save the maintenance programmer from needing to examine the implementation to see how it works. When there is a standard like this through the entirety of the code base, the amount of time saved by not having to study implementations is huge.

Toward the end of achieving this goal, one idea I had was a “naming” audit. This activity would consist of the team assembling for an hour or so, perhaps over pizza one lunch or evening, and going through the code base looking at all of the names of methods, variables, and classes. Any names that weren’t accurate or sufficiently descriptive would be changed by the group to something that was clear to all. I think the ROI on this approach would be surprisingly high.

But if you can’t do that, maybe a more distributed approach would work — one that combines the best elements of shaming and good-natured ribbing, like having the person who breaks the build buy lunch. So maybe any time you encounter a poorly named method in the code, you rename it to something ridiculous to underscore the naming problem that preceded it. You bring this to the method author’s attention and demand a better name. Imagine seeing your methods renamed to things like:

  • ShockAndAwe()
  • Blammo(int x)
  • Beat(int deadHorse)
  • Onoes(string z)
  • UseTheForceLuke()

I’m not entirely sure if I’m serious about this or not, but it would make for an interesting experiment. Absurd names kind of underscores the “dude, what is this supposed to mean” point that I’m making, and it’s a strategy other than endless nagging, which isn’t really my style. I’m not sure if I’ll try this out, but please feel free to do so yourself and, if you do, let me know how it goes.


The Joy of Adding Code to Github

These days, a good portion of my time is spent on overhead tasks: management tasks, strategy meetings, this sort of thing. In my role, I also am responsible for the broad architectural decisions. But what I don’t get to do nearly as often between 9 and 5 these days is write code. All of the code that I write is prototype code. I’m sure that I’m not unique in experiencing drift away from the keyboard at times in my career — some of you do too as well. As you move into senior or lead type roles and spend time doing things like code reviews, mentoring, etc, it happens. And if you really love programming — making a thing — you’ll feel a loss come with this.

Of course, feeling this loss doesn’t have to be the result of having more overhead or leadership responsibilities. It can happen if you find yourself working on a project with lots of nasty legacy code, where changes are agonizingly slow and error prone. It can happen in a shop with such intense and byzantine waterfall processes that most of your time is spent writing weird documents and ‘planning’ for coding. It can happen in an environment dominated by loudmouths and Expert Beginners where you’re forced by peer pressure or explicit order to do stupid things. It can happen when you’ve simply lost belief in the project on which you’re working or the organization for which you’re working. And it almost certainly will happen to you at some point in your career.

Some call it “burnout,” some call it “boredom” and some call it “losing your way.” It has a lot of names. All of them mean that you’re losing connection with the thing you’ve liked: programming, or making a thing out of nothing. And you need that. It’s your livelihood and, perhaps more importantly, your happiness. But, you can get it back, though, and it isn’t that hard.


Recently, I decided to order a few more home automation pieces (blog to follow at some point) and reboot the design of my home automation server to make use of a Raspberry Pi and a REST service in Python. I don’t know the first thing about Python, but after the parts arrived, and I spent a few hours reading, poking around, failing, and configuring, I was using Fiddler to turn all the lights off downstairs (code on github now — work in progress).

There is nothing quite like the feeling of creating that new repository. It’s daunting; I’m about to start learning a new programming language and my efforts in it will most certainly be daily-WTF-worthy until I learn enough to be passingly idiomatic in that language. It’s frustrating; it took me 15 minutes to figure out how to reference another code file. It’s tiring; between a day job taking 50+ hours per week and the work I do with my blog and Pluralsight, I’m usually coding at midnight on school nights. But forget all that because it’s exhilarating ; there’s nothing like that feeling of embarking on a journey to build a new thing. The code is clean because you haven’t had a chance to hose it up yet. There is no one else to tell you how, what, or when because it’s entirely your vision. The pressure is off because there are no users, customers, or deadlines yet. It’s just you, building something, limited only by your own imagination.

And what a powerful feeling. If you’ve got a case of the professional blues, go out and grab this feeling. Go dream up some new project to start and chase after it. Recapture the enjoyment and the excitement of the desire to invent and create that probably got you into this stuff in the first place.