Stories about Software


RESTful Home Automation

Here are the general steps to get a REST service going on your Raspberry Pi, using Python. One thing that I’ve learned from blogging over the last several years is that extremely detailed, granular how-tos tend to be the most yawned-at posts. So this is just a quick overview of how you can accomplish the goal without going into a lot of detail. If you want that detail, you can drill into the links I’m providing or else feel free to ask questions in comments or via email/twitter.

  1. Go out and buy these things: USB transceiver, plugin transceiver, lamp module (optional)
  2. On your Pi, install apache with sudo apt-get install apache2.  This is the web server.
  3. Also on your Pi, install web2py with sudo apt-get install python-webpy.  This is the module that makes setting up a REST service a snap.
  4. Install some driver dependencies (I will probably later roll these into what I’m doing) with “sudo apt-get install libusb-1.0 python-usb”.  Here are more detailed instructions from the page of the home automation python driver that I’m using.
  5. Follow the instructions on that page for disabling interfering kernel drivers.
  6. Give your user space account permissions to hit the USB device from the referenced instruction page, but note a typo where he says “sudo nano /etc/udevrules.d/cm19a.rules” and you really want
    “sudo nano /etc/udev/rules.d/cm19a.rules”.
  7. Now go get my stuff from github and run a web server using python rest.py <port>

That’s all there is to it.  Right now, at the time of writing, you would go to http://<your pi’s ip>:<port>/office/on to basically turn everything from A on.  A and office are both hard-coded, but that’s going to change in the next few days as I grow this service to support adding rooms and lights via PUT, and storing them as JSON documents on the server.  You’ll be able to add a light with PUT, supplying the room, light, and X10 code, and then you’ll subsequently be able to toggle it with http://pi:port/room/light/{on/off}

You can also just install Andrew’s driver and use it as-is.  It even has a web mode that supports query parameters.  The reason I didn’t do things that way is because (1) I wanted a REST service and (2) I wanted to be able to customize everything as I went while learning a new language.


Getting Started with Raspberry Pi

What and Why?

(If you’re here because you googled for a RPi tutorial, I’d suggest skipping to the next section)

I’ve recently throttled back on being all .NET all the time, mainly because I’m trying to spend more time working on my home automation. Toward that end, I recently picked up a Raspberry Pi. Since the organization’s home page neglects to include an elevator pitch as to what it actually is, I’ll summarize by saying that the Raspberry Pi (or RPi, for short) is a fully functional computer that’s about the size of a credit card and contains a good cross section of the device I/O that you might want, including audio, HDMI, USB, and ethernet. Oh, and they’re $35.

I’d imagine that most of you reading have either heard of this or, if you haven’t, you’re thinking of half a dozen uses now that you’ve read my description. But if you aren’t, here are a few uses you might find attractive: tiny media PC, small robot controller, car OBD troubleshooter device, Roomba brainwasher (I totally want to do this and use it to chase my cats around). One could also use it, as I plan to, for home automation.

My purpose here with my first RPi is going to be relatively simple — I’m going to offload the home automation controlling functionality of my home’s server onto it. The server I’m currently using is a repurposed dinosaur that I took with me to college (P2, 400 MHz processor, and tricked out to the max in slots with a whopping 3 128 MB sticks of RAM). It chugs along with an old Fedora installation, running a web server, SVN, a few databases, samba, and a few other goodies, as well as being hooked up to my big external drive where media is stored and eventually whisked into the cloud via Crash Plan. Add to this the fact that there’s a firecracker module plugged into its serial port to make the jump from my software to the electrical, and this is a pretty busy old machine that may not be long for this world. So I’d like gradually to parcel out its functionality to a handful of other machines to minimize my exposure to an extremely annoying failure, and the RPi is perfect, spec-wise and form-factor-wise, for the home automation web server.

Before Getting Started

So your Pi arrived in the mail, it has no installed OS, no instructions, and you’re not really sure what to do. No worries. I was like you, and I fumbled through it cobbling together a variety of guides and tutorials, and here, centralized in one place, is how I did it. This is the dead simple, option-free way to get something going with your RPi. All you’re going to get from this tutorial is an OS and web server on the device, but you’re not going to have to pick

RaspberryPiFirst off, prerequisites. To get started, here’s what you need:

  1. The RPi, obviously (pictured)
  2. A formatted Mini SD card that will act as the hard drive (pictured), or optionally a MicroSD card with a mini adapter. Either way, it needs at least 2 GB of space
  3. A micro-USB electronic device charger (i.e. an “anything but Apple” cell phone charger) to power the Pi.
  4. A desktop, Windows computer with either an onboard or connected SD card reader/writer
  5. An internet connection
  6. (Optional) HDMI to DVI adapter if you plan to spend a significant amount of time with your RPi plugged into a monitor.

You can probably make it happen without most of these things, but this is what I was working with when I did it, so if you have a Mac or want a different Linux distro on your RPi, this isn’t the tutorial for you.

Linux on the Pi

Here are the steps that I followed:

  1. First, download the zipped OS, “Raspbian Wheezy” for the RPi at this link, which will actually trigger the download. If that becomes defunct or you want to find it on your own, you can go to the RPi download site.
  2. Now that you have the OS, you need a means of putting it on the SD card. For that, you’ll need the Win32 Image writer (link triggers actual download of zip) or find the download yourself at their site.
  3. Plug your SD card into your SD reader/writer.
  4. Extract the contents of both of the zip files that you downloaded and launch the disk manager utility with administrator rights. If it takes a long time to load, there’s probably a driver issue with your SD card — that happened to me, but the fix was very specific to my hardware and is thus beyond the scope of this tutorial.
  5. Choose the Wheezy-Raspbian img file and the drive in which your SD card is mounted on your computer (below). For me the only option was the drive with the SD card, but make sure you double check that it’s the right one — I don’t want to be responsible for your frying some drive you need with a Linux install.WinDiskManager
  6. When you’re sure you’re not blowing anything you need away, click “Write,” and then click “OK” when warned.
  7. The write process will take a while, with specifics depending on your hardware. For me it took about 10 minutes. At the end, a “Write Successful” dialog will appear.
  8. Pop the SD card out of the computer and put it into the RPi.
  9. Plug your RPi’s HDMI into a TV (or monitor with the adapter) and plug a USB keyboard and mouse into the RPi.
  10. Now plug the RPi into power and let it boot up. It should look like a normal Linux boot sequence:
  11. Once it boots up, you’ll be shown some initial configuration options. I enabled SSH and expanded the root partition to use my entire 8 Gig SD card. Your mileage may vary here, but I bought the SD card just for the Pi and I have no interest in leaving the Pi plugged into my bedroom TV on a permanent basis, preferring to remote in.
  12. I also enabled booting into desktop, though I may later get rid of that option since the Pi will be headless. From there I selected “Finish” and rebooted. This took a while since it resized the root partition on reboot.
  13. Once it booted, it booted to a cute little desktop with the RPi logo in the background and you can configure it like any other Linux distro.
  14. One early thing to note is that the default user/pass is pi/raspberry — this will help if you drop into a console tty. (i.e. you hit ctrl-alt-f4)
  15. On the desktop, I noted that wheezy ships with WPA-supplicant for wifi config, which is handy if you have a wifi USB dongle, though I did not go this route — I have known much pain from using wpa_supplicant 5-6 years ago with AES before the utility was mature.
  16. After playing with the RPi desktop a bit, I powered down and disconnected from the TV, setting it up headless in my office, to be accessed via SSH.
  17. At this point, I tunneled in via SSH and ran an update. I had seen that things weren’t working on the desktop because it needed updating, so, first things first. “sudo apt-get update” did the trick (after about 4 or 5 minutes).
  18. To put apache on there, I did “sudo apt-get install apache2″. Ah, apt-get, the original app store.
  19. Finally, if you’re going to have a web server, it makes sense to give it a static IP. There’s nothing specific to Wheezy about this — it’s just standard Linux networking involving modifications to the network/interfaces file.

And, viola! The transformation from RPi in the box it shipped in to little web server is complete.


Configuring Fedora and MongoDB

In my last post, I covered installing MongoDB on Fedora. This is another post as much for my reference as for anything else, and I’ll go over here getting set up so that my application successfully uses the MongoDB server.

When I left off last time, I had successfully configured the server to allow me to create documents using the mongo command line utility. So, I created a collection and a document and was ready to access it from my web application. Following the examples in the MongoDB java tutorial, I created the following implementation of my HouseService interface that I had previously hard coded room values into:
Read More


Setting up MongoDB on Fedora

This is one of those posts as much for my own reference as anything else, and if it helps others, then bonus.

I installed MongoDB on my old Fedora server just now, and while it was actually pretty straightforward to do, I figured I’d document the steps. The first thing to remember is that there are two separate installs: the client and the server (this is what tripped me up on the MongoDB site’s documentation). There isn’t some deal where running server install also gives you a client.

So, to set up a smooth, easy install with yum, the first thing you need to do is create the file “/etc/yum.repos.d/10gen.repo”. Once you’ve created that file, open it and add the following:

Now, once you’ve got that in place, run the command, “yum install mongo-10gen mongo-10gen-server”. This will install both the client and the server. When the client and the server are in place, you can start the server by running “/etc/init.d/mongod start”. Finally, if you’re like me, you probably want the server to run automatically. If that’s the case, execute the command “chkconfig mongod on”.

And, you’re set. MongoDB server is installed and running and will also run on your next reboot.


Fixing the Crackle and Pop in Ubuntu Sound

So, as I blogged previously, I’ve been re-appropriating some old PCs and sticking Ubuntu 11.10 on them. I had one doing exactly what I wanted it to do in the context of my home automation network with one small but annoying exception. Whenever I would play music through headphones or speakers, there would be a crackling noise. A google search turned up a lot of stuff that I had to weed through, but ultimately, this is what did the trick:

I opened /etc/pulse/default.pa and found the line

load-module module-udev-detect

This was located in an “ifexists” clause. It should be replaced by this:

load-module module-udev-detect tsched=0

From there, reboot. I can’t speak to whether this will fix everyone’s crackle or just some people’s, and I can’t speak to why this setting isn’t enabled by default, but c’est la vie. This is what did the trick for me, and hopefully it fixes the problem for someone else as well.


Ubuntu and Belkin Dongles Revisited

Previously, I posted about odyssey to get belkin wireless dongles working with Ubuntu. Actually, the previous post was tame compared to what I’ve hacked together with these things over the years, including getting them to work on Damn Small Linux where I had to ferret out the text for the entire wpa_supplicant configuration using kernel messages from demsg. But, I digress.

I’m in the middle of creating an ad-hoc “music throughout the house” setup for my home automation, and this involves a client computer in most rooms in the house. Over the years, I’ve accepted donations of computers that range in manufacture date from 1995 to 2008, and these are perfect for my task. Reappropriated and freed from their Windows whatever, they run ably if not spectacularly with XUbunutu (and, in some cases DSL or Slackware when that’s too much for a machine that maxes out at 64 meg of RAM).

So, I have this setup in most rooms, and I just remodeled my basement, which was the last room to get the setup. I had one of these things working with the dongle and everything, but the sound card was this HP Pavilion special that was integrated with a fax card or something, and the sound just wasn’t happening. So, after sort of borking it while trying to configure, I scrapped the effort and reappropriated an old Dell.

Each time I do this, I grab the latest and greatest Ubuntu, and this time was no different. Each time, I check to see if maybe, just maybe, I won’t have to pull the Belkin drivers off of the CD and use ndiswrapper, and lo and behold, this was the breaking point – I finally didn’t.

I wish I could say it worked out of the box, but alas, not quite. I plugged in the dongle and the network manager popped up, and sure enough it was detecting wireless networks, but when I put in all of my credentials, it just kept prompting me for a password. I remembered that Network manager had difficulty with these cards and WPA-PSK security protocol, so I tried another network manager: wicd. Bam! Up and running.

So, for those keeping score at home, if you have Ubuntu 11.10 (Ocelot) and a belkin dongle, all you need to do is:

sudo apt-get install --reinstall wicd
sudo service network-manager stop
sudo apt-get remove --purge network-manager network-manager-gnome
sudo service wicd restart

And, that’s it. You should be twittering and facebooking and whatever else in no time.


Since making this post, I set up another machine in this fashion, and realized that I made an important omission. The wicd wireless setup did not just work out of the box with WPA2. I had to modify my /etc/network/interfaces file to look like this:

auto lo
iface lo inet loopback

auto wlan0
iface wlan0 inet static
address {my local IP}
wpa-driver wext
wpa-ssid {my network SSID}
wpa-ap-scan 2
wpa-proto WPA RSN
wpa-pairwise TKIP CCMP
wpa-group TKIP CCMP
wpa-key-mgmt WPA-PSK
wpa-psk {my encrypted key}

For my network, I use static IPs and this setup was necessary to get that going as well as the encryption protocol. Without this step, the setup I mentioned above does not work out of the box — wicd continuously fails with a “bad password” message. Adding this in fixed it.



A Small, Functional Makefile

I don’t write C++ all that often these days, but I suppose that I spent so many years with it that it never really feels foreign to me when I come back to it. What does oftentimes feel foreign is generating a Makefile when developing in Linux. I’ve made enough of them over the years that I know what I’m doing but not so many that I can go off the cuff after six months or a year break from them.

So I’m sticking a sample Makefile here. This is somewhat for me to refer back to whenever I need to, but I’ll also explain some of the basics. In this example, I’m creating a little C++ application for calculating the odds of poker hands, given what is on the table at the moment. At the time of writing, the example, in its infancy, has only one class: Card. So the files at play here are card.h, card,cc and main.cpp. The main class references card.cpp, which, in turn, references its class definition header file, card.h.

So there’s the simple Makefile. If you take a look at this, the main purpose of the Makefile is, obviously, to compile the source, but also to automate linking so that, as projects grow, you don’t have increasingly unwieldy g++ command line statements. So we define a few Makefile rules. First, card.o is generated by compiling card.cc. Second, main.o is generated by compiling main.cpp. The executable is generated by linking the two object files, and “all” is the executable.

That’s all well and good, but I can eliminate some duplication and make this more configurable. I’ll use Makefile variables so that I don’t have to repeat things like “card,” “oddscalc,” and “g++” everywhere.

In addition, I can see the inevitable redundancy coming from our previous Makefile. As soon as I add hand.cpp/hand.h and deck.cpp/deck.h, I’m going to have to create rules for them as well. Well, I don’t want to do that, so I’m introducing a scheme that, in essence, says, “compile every .cpp file I give you into a .o file and link it in the main assembly.” This will be expressed with a “.cpp.o” rule.

With this Makefile, if I want to add a new class, all I need to do is add the class’s .cpp file to the “SOURCES” definition line and it will get compiled and linked for the application. (Well, obviously, I need to write the class as well, but we’re just talking about the Makefile here.)

So that’s it. There are a lot of things you can do with Makefiles. Some people create a variety of build configurations. “make tar” is a popular option as well. But I think that this Makefile is relatively simple and elegant, and it’s easy to add to.


New Ubuntu and Weird Old Sound Cards

In an earlier post, I described how one can go about installing a Belkin USB dongle and a very recent version of Ubuntu on a dinosaur PC. Tonight, I’m going to describe a similar thing, except that instead a new piece of hardware, it’s a dinosaur that came with the PC itself. I must admit, this is more likely to be useful to me the next time I encounter this on an old PC than it will be to anyone else, but, hey, you never know.

First, a bit of back story here. As I’ve alluded in previous posts, one of my main interests these days is developing a prototype affordable home automation system with the prototype being my house. So far, I have a server driving lights throughout the house. This can be accessed by any PC on the local network and by my Android phone and iPod. The thing I’m working on now is a scheme for playing music in any room from anywhere in the house. Clearly the main goal of this is to be able to scare my girlfriend by spontaneously playing music when I’m in the basement and she’s in the bedroom, but I think it’s also nice to be able to pull out your phone and queue up some music in the kitchen for while you’re making dinner.

Anyway, one of the cogs in this plan of mine is reappropriating old computers to serve as nodes for playback (the goal being affordability, as I’m not going to buy some kind of $3000 receiver and wire speakers all over the house). I should also mention that I’m using Gmote server for the time being, until I write an Android wrapper app for my web interface. So, for right now, the task is getting these computers onto the network and ready to act as servers for “play song” instructions.

The computers I have for this task are sort of strewn around my basement. They’re machines that are so old that they were simply given to me by various people because I expressed a willingness to thoroughly wipe the hard drive and I’m the only person most people know that’s interested in computers that shipped with Windows 98 and weigh in with dazzling amounts of RAM in the 64-256 megabyte range. These are the recipients of the aforementioned Ubuntu and Belkin dongles.

So, I’ve got these puppies up and humming along with the OS and the wireless networking, and I was feeling pretty good about the prospect of playing music. I setup Gmote, and everything was ready, so I brought my girlfriend in for my triumphant demonstration of playing music through my bedroom’s flatscreen TV, controlled purely by my phone. I plugged in the audio, queued up Gmote, and everything worked perfectly–except that there was no sound. My phone found the old computer, my old computer mounted the home automation server’s music directory (itself mounted on an external drive), Gmote server kicked in… heck, there was even some psychedelic old school graphic that accompanied the song that was playing on the VGA output to the flat screen. But, naturally, no sound.

So, I got out my screwdriver and poked around the internals of the old computer. I reasoned that the sound card must be fried, so I pried open another computer and extracted its card, and put everything back together, and viola! Sound was now functional (half a day later, thus taking a bit of the wind out of my grand unveiling’s sails). So, I pitched the sound card and moved onto getting the next PC functional. This PC had the same sound card, and I got the same result.

I smelled a rat, reasoning that it was unlikely that two largely unused sound cards were fried. After a bit of investigation, I discovered that the problem was the card in question was an ESS ES169, which is actually a plug and play ISA device and not a PCI device. I had reasoned the previous card was fried when I didn’t see it in BIOS PCI list. But, there it was in BIOS ISA list. Because, naturally, a sound card inside of the computer, like a printer or USB external hard drive, is a plug-and-play device.

But anyway, with that figured out, I was all set… kind of. It took me an hour or two of googling and experimenting to figure it out, but I got it. I had to experiment because this card was pretty dated even five years (or roughly the 438 version of Ubuntu) ago, and so I wasn’t dealing with the same utilities or configuration files.

So anyway, with the grand lead up now complete, here is the nitty gritty.

When you boot into Ubuntu, it, like me or any other sane entity, has no idea what this thing is. You’ll see nothing in lspci about it, of course, and if you sudo apt-get install the toolset that gives you lspnp, you’ll see it as an unknown device. Don’t let that get you down, though; it was, at some time, known to someone. The first thing to do is use sudo and your favorite text editor to modify /etc/modules. You’re going to add “snd-es18xx” to that file and save it.

Next, add the following text to the configuration file “/etc/modprobe.d/alsa-base.conf”:

And that’s that. Now, if you reboot you should see a working audio driver and all that goes with it. You can see that it’s working by playing a sound, or by opening up volume control and seeing that you no longer are looking at “Dummy Output” but a realio-trulio sound output.

I confess that I don’t know all of the exact details of what that configuration setup means, but I know enough mundane computer architecture to know that you’re more or less instructing this device on how to handle interrupts like the PCI device it’s pretending to be and Ubuntu thinks it ought to be.

I’d also say that this is by no means a high-performance proposition. I’d probably be better served to get a regular sound card, but there’s just something strangely satisfying about getting a twelve-year-old computer with a stripped down OS to chug along as well as someone’s new, high powered rig that’s been loaded down with unnecessaries. I suppose that’s just the hopeless techie in me.


Old Linux Computer with new Wireless Encryption

As I’ve previously mentioned, one of the things that I spend a good bit of time doing is home automation. For prototyping, I have a bunch of very old computers that I’ve acquired for free. These are scattered around the house (much to the dismay of anyone with a sense of decor) and I use them as dumb terminals for interfacing with the home automation system. That is, I have a client/server setup, and these guys put the “thin” in thin-client.

Now, because many of these were made in the 90s, I don’t exactly put Windows 7 (or even Windows XP) on them. Most of them range between 64 and 256 megs of RAM and are of the P-series intel processors from that era. So, the natural choice is Linux. I have had luck using Damn Small Linux (DSL), TinyCore linux, Slackware, and Ubuntu. Since most of these are not for the faint of heart or anyone who isn’t comfortable editing low level configuration files in pico or VI, I’ll focus this post on Ubuntu (more specifically, Xubuntu, since the standard windows manager is a little much for these machines).

Because of the nature of what I’m doing–allowing machines on my network to control things in the house like lights and temperature–network security is not a convenience. It has to be there, it has to be close to bulletproof, and I can’t simply say “the heck with it — I’ll compromise a little on the settings to make configuration easier.” So I use a WPA-PSK encryption scheme with a non-broadcasting network.

Now, my house has three stories including the basement, and while I enjoy home improvement, I’m not such a glutton for punishment that I’ve retrofitted cat-5 connections in every room. Getting these old Linux machines connecting to the network is an interesting problem that I’ve ultimately solved by buying a series of Belkin wireless USB dongles. For the most part, these old computers do have a couple of USB jacks. So what I’ll document is how I’ve had success setting up the networking.

The first thing I do after installing the Xubuntu OS is to go to my main office computer and download ndiswrapper. This link is helpful, as it points you to where you can go about downloading the debian package to install from the command line: Ndiswrapper.. Ubuntu OS generally assumes for the purpose of their package manager (which, as an aside, makes me smile every time someone says that the Android/Apple walled app garden is a newfangled concept) that you have an internet connection or that the CD has the packages that you need. If I had the former, this post would be moot, and the nature of the slimmed-down Xubuntu install precludes the latter.

So, you can find the version for which you need ndiswrapper and grab it from the mirrors. From there, you can install the packages by following the instructions at the link for using dpkg from the command line. After doing so, you will be equipped with everything you need from ndiswrapper. Ndiswrapper is a cool little utility that essentially inserts a logical layer between the drivers and Linux, thus allowing Linux to use Windows drivers as if they were native to that OS. The FOSS folks are generally cool this way — no one writes anything with Linux in mind, so they bend over backwards to be compatible.

Once you have ndiswrapper installed, the next thing to do is to grab the CD that came with the Belkin dongle and pop it into the Linux machine. Mount the CD (if it doesn’t automount — I tend to do all this from TTY4 rather than the UI because when you only have a few hundred meg of RAM, the UI is a little slow) and navigate to the folder containing the .INF file. If you’re doing anything like I am, this is going to be inside of a folder with a name like WinXP2000. The key thing here is to be sure that you find the driver that matches your processor architecture — probably i386. This can easily be accomplished if you know what version of Windows came installed on the machine before you wiped it to put Linux on. If the machine didn’t initially come with Windows, you probably know what you’re doing anyway.

From here, you can execute a “sudo ndiswrapper -i {yourfile}.inf”. This will install the driver in the configurables of the ndiswrapper utility and ndiswrapper should take care of loading it on your next and any subsequent reboots. While you’re at it, you may as well reboot now and get the driver loading. If you’re feeling intrepid, you can try restarting the networking service to see if you start to connect, but I make no guarantees that this will work.

Once you’ve rebooted, Linux should recognize the driver, but you won’t be connecting to your network. I’m not sure off the top what it loads for default settings, but it sure isn’t a requester configured for encrypted access to your network. So now, I edit my /etc/network/interfaces file to look like the following:

If you fill in your own info for the {}, you should be set to go. This will configure you as a supplicant (connecting client) to a network with WPA/PSK, static rather than DHCP addresses, and non-broadcasting status (though this doesn’t really matter on Linux — iwlist, the linux utility, sees networks whether or not they broadcast). And, best of all, it will do all of this when you boot since it’s part of the interfaces file. No adding things to rc.local or your login script like in the old days.

The only extra thing here is generating your PSK. That is a little beyond the scope of what I’m explaining here, but if there is some interest in the comments for this post, I can create a follow up explaining how to do that.

I’m not sure how many people are fellow enthusiasts of re-appropriating old clunker machines to do cool, new things, but I hope this helps someone, as these sorts of configuration issues can be maddening.

Acknowledgements | Contact | About | Social Media