Since I had been working on arduino, I happened to have one of those 3.3V FTDI USB-serial converters, and was able to make a cable that connects my bedside Zeo to my server closer via a custom cable going through my existing Cat-5 wiring.

The example code they gave is however, while very fancy, not that simple to borrow from for a more simple application that simply needs to keep track of the sleep stage.

So, after finding the relevant info, I compiled the code below, which I'm posting for others to use (you can also download it here: zeo raw data code sample).

#!/usr/bin/python

# This is based off the Zeo Raw Data Library at the bottom of
# http://zeorawdata.sourceforge.net/starting.html
# This script is a much simpler version of how to log basic data than the cool
# and pretty GUI_Viewer from http://zeorawdata.sourceforge.net/examples.html
# This script is an easier example to steal from for simple logging/integration.

# By Marc MERLIN <marc_soft@merlins.org> / 2011/12/17
# License: GPLv3.

#gandalfthegrey [mc]$ ./logsleep.py
# 2011-12-17 12:19:10: HeadbandDocked
# 2011-12-17 12:19:15: HeadbandUnDocked
# 2011-12-17 12:19:22: Sleep state: Undefined
# 2011-12-17 12:19:52: Sleep state: Undefined
# 2011-12-17 12:20:22: Sleep state: Undefined
# 2011-12-17 12:20:52: Sleep state: Undefined
# 2011-12-17 12:21:22: Sleep state: Undefined
# 2011-12-17 12:21:52: Sleep state: Undefined
# 2011-12-17 12:22:22: Sleep state: Undefined
# 2011-12-17 12:22:52: Sleep state: Undefined
# 2011-12-17 12:23:22: Sleep state: Undefined
# 2011-12-17 12:23:52: NightStart
# 2011-12-17 12:23:52: Sleep state: Awake
# 2011-12-17 12:24:22: Sleep state: Awake
# 2011-12-17 12:24:52: Sleep state: Awake
# 2011-12-17 12:25:22: Sleep state: Awake
# 2011-12-17 12:25:52: Sleep state: Awake
# 2011-12-17 12:26:23: Sleep state: Awake
# 2011-12-17 12:26:32: HeadbandDocked

# System Libraries
import time
import sys

# Zeo Libraries
from ZeoRawData import BaseLink, Parser
from ZeoRawData.Utility import *

# User customizable variables
port = '/dev/ttyUSB0'

class Callbacks:
    def logtime(self):
	return time.strftime("%Y-%m-%d %H:%M:%S: ", time.localtime())

    def SliceCallback(self, slice):
        if slice['SleepStage']:
	    print self.logtime() + "Sleep state: " + slice['SleepStage']

    def EventCallback(self, logtime, version, event):
	print self.logtime() + event

if __name__ = "__main__":
    link = BaseLink.BaseLink(port)
    parser = Parser.Parser()

    callbacks = Callbacks()

    # Add Callbacks and Start the Link
    link.addCallback(parser.update)
    parser.addSliceCallback(callbacks.SliceCallback)
    parser.addEventCallback(callbacks.EventCallback)
    link.start()

    # Infinitely loop to allow the script to run continuously
    while(True):
        time.sleep(5)

that's cute :)

Vancouver if a nice city to go visit, so that was not a bad thing at all :) and I was invited this year to talk about power monitoring. Outside of a small issue where I was so busy before the conference that I forgot which day I was suppose to fly there, and missed my flight to Vancouver, I got there the next morning on some very early flight and things worked out (putting aside the resulting poor sleep).

My talk on power monitoring, while having taken way too long to write, went well. Here is the link to the slides on the Power Monitoring with Linux talk.

A few keynote pictures:

A couple of random slides:

The 20 year party I was very underdressed for :)

And the random picture:

Despite the travel mishaps, it was a good conference.

I had an old CM19a (USB X10RF and X10Sec transceiver) lying around. This was more desirable than the well known CM26a used by many misterhouse users in that on top of being USB, it more importantly can decode X10 Security RF signals, as well as send X10RF signals too.
Now, the problem with the CM19a, is that like the CM26a, it has a useless antenna and therefore a useless RF range. The good news however is that the same antenna hack that can be applied to the CM26a works with the CM19a too.

For pictures, see the example for the CM26a antenna modification this hack was based on (scroll to the antenna plug wiring).

In a nutshell, you cut the antenna wire to go far enough to reach the new antenna plug that you attach to the plastic (I used my soldering iron to burn a hole through it). The antenna wire will plug to the center connector. Then, the tricky side is to use the piece of wire you cut off, and connect it to the ground plate of that board. Ground is actually easy to solder to: most holes through the board with metal on each side are ways to pass ground from one side of the board to the other. I used one of them (see red arrow) to solder my other wire to, and connected that to the ground of the new plug.

You then have the option of using a dipole antenna or a quarter plane antenna to connect to your new plug (which one is best depends on where you put the antenna and what kind of area you are trying to cover).

By now, you actually have turned your CM19a into a device that's almost as good as a W800 for reception, but with the bonus of being able to send data too.

Next (for me), was making use of this in misterhouse. Because it is a USB device that does not emulate a serial port, it will not work in misterhouse without a special driver.
To find the simplest way to solve my problem, I decided to use the open source mochad to talk USB to the device and spit out the data frames it was receiving (I hacked mochad to spit out undecoded data). I then wrote a glue shell script that sends that data to a pipe which misterhouse can then read from. It then sends that data into the misterhouse X10_RF module, reusing the common decoding and injection code used by the X10_W800 and X10_MR26 misterhouse modules.
This all ended up in the new lib/X10_CMxx.pm module I wrote and added to misterhouse svn.

You can find more details on the Misterhouse Page for X10Sec and X10RF support with CM19a and CM15a through mochad.

Here are two examples of logs with debugging enabled:

02/07/2011 14:51:45  CMxx: Ignoring first send of X10RF data from mochad (looking for confirmation resend): 8F 80 84 7B F1 80
02/07/2011 14:51:45  CMxx: decoded data received from mochad: 07/02 14:51:45 Rx RFSEC Addr: 8F:F1:80 Func: Contact_normal_min_DS10A
02/07/2011 14:51:45  W800: security: unmatched device 0xf1 (state = NormalMin)
02/07/2011 14:51:45  CMxx: X10RF data from mochad: 8F 80 84 7B F1 80
02/07/2011 14:51:45  X10_CMXX: security: unmatched device 0xf1 (state = NormalMin)
02/07/2011 14:51:45  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 2): 8F 80 84 7B F1 80
02/07/2011 14:51:46  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 8F 80 84 7B F1 80
02/07/2011 14:51:46  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 8F 80 84 7B F1 80

02/07/2011 14:51:59  CMxx: Ignoring first send of X10RF data from mochad (looking for confirmation resend): 60 9F 20 DF
02/07/2011 14:51:59  CMxx: decoded data received from mochad: 07/02 14:51:59 Rx RF HouseUnit: A1 Func: Off
02/07/2011 14:51:59  XA1AK: testx10 off
02/07/2011 14:51:59  CMxx: X10RF data from mochad: 60 9F 20 DF
02/07/2011 14:51:59  XA1AK: testx10 off
02/07/2011 14:51:59  XA1AK: testx10 off
02/07/2011 14:51:59  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 2): 60 9F 20 DF
02/07/2011 14:51:59  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF
02/07/2011 14:51:59  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF
02/07/2011 14:51:59  CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF

Ultimately, mh without debugging only sees, which is what you want:
02/07/2011 14:51:59  XA1AK: testx10 off

Paulway and Angus Lees

For the first time, I went to the arduino miniconf on monday, since I had been meaning to learn more about arduino, and that was a good excuse to do so. So, I built a mobsendat arduino board with accelerometer, GPS, temp sensor, pressure sensor, and more:

Arduino build miniconf

The kit

My kit, all built

Most people built the kit for rockets they built

Here are a few slides from keynotes and talks I went to:

IPv6 keynote. Newsflash: we're screwed...

Tux sent to space on a weather balloon

Eric Allman talk

Jon had a very fun talk about using the Kinect to control a self levelling quad-copter

Paul's talk on the wakelock discussions on lkml

The enjoyable, if not controversial keynote on personal data and cloud services

My talk on misterhouse and home automation

And of course, LCA is also fun for various socializing and daily dinners, starting with the speaker Dinner:

Professional Delegates Dinner:

Penguin Dinner:

All in all, a fun year was had again, a big thanks go to the organizers for pulling it out.

Vmware linux kernel support downfall

I need to start by saying that I've always considered vmware an impressive product. Some of what they're doing is incredibly hard (fast VT without CPU support) and they're doing it very well.

Yet, I've always had a gripe with Vmware which finally got me to switch to Virtualbox just now: their total lack of support for new kernels as they come out.
While it would be nice, I'm not asking for their website to have up to date patches for the latest rc kernel patch that just came out 2 hours ago, but they have always lacked support for new stable kernels, even months after they came out (sometimes up to six months almost!).

It's not like they don't have kernel engineers there, why don't they write timely patches to vmmon, vmnet and friends a few days after each new stable kernel comes out? Ok, there is the support and testing issue, but then just post them unofficially on the web site as 'unsupported', which is better than my googling for the same thing off some random guy's web page.
Now you could say "just upgrade to workstation 7.x", but shouldn't I get kernel support for the existing workstation license I already paid for, for a reasonable number of years, like 3 to 5? I'm not asking to have vmware 2 working anymore... Forcing me to buy the next vmware every time just to get kernel support is wrong. The worst part is that even workstation 7 still does not support kernel 2.6.35 or 2.6.36, so even by paying for the next version, I'm still unsupported.

For "fun", I tried to get the lastest vmware workstation from their web site, take its modules, patch them for 2.6.36 since they do not work as is, and then try to use them with vmware 6.5. Unfortunately, they're not backward compatible :(

So, I'm left with using the free limited vmware player when I had 5 paid for licenses for vmware 2, 3, 4, 5, and 6 (!). That's just wrong: I shouldn't have to use the free limited product when I have paid licenses for multiple older vmware versions with no kernel support...
And to add insult to injury when I installed the latest vmware player, it removed vmware workstation from my system without asking me, never mind if I wanted to reboot with an older kernel and use my older vmware workstation. Grr, that's not cool guys!

So, while I really liked vmware, the company and the product, this just drove me away and enticed me to try VirtualBox.
Sorry to tell you vmware: you lost a customer who could otherwise recommend your products for server room use because you couldn't be bothered to support new linux kernels timely as they came out in the 10 years or so you've been in business and I've used your products, and you clearly dropped support for your paid customers as soon as a new vmware came out, forcing people to upgrade and pay for a new version.

VirtualBox

Because USB support is a requirement for me (I mostly use windows to sync to hardware that only has windows support, like garmin GPSes), I had to use the closed source edition from Oracle.

First impressions were good: Oracle provides multiple native packages for all popular distributions (which is better than the vmware bundle that decides to uninstall old versions of vmware products without asking me), and one apt-get command later, I had virtualbox running on my laptop.

The part that really impressed me was that VirtualBox was able to re-use my vmware disk image cut in 2GB chucks as is. I just pointed it to my vmware image, and it just booted windows. Now, things were a bit ugly until I got its tools installed (a bit like vmware tools), once that was done, things worked pretty well. I read some notes about the fact that a WinXP image might not work as is in virtualbox due to ACPI and IOAPIC, but my W2K image from 10 years ago just worked as is.
A small problem was that it was incompatible with the AMD PCnet emulation than vmware used, so networking didn't quite work until I switched virtualbox to emulate an e1000 card and installed the e1000 driver in my vmware image. After that, things were good.

I was then even able to shut down the image, and re-open it in vmware player 7 (the only thing to be very careful about was that vmware player remembered that the image was a snapshot and tried to restore windows to its running state in vmware with disk blocks that has since changed due to Virtualbox having run in the meantime. Worryingly enough I did not find a way to tell vmware player to start the image from scratch as opposed to trying to restore the frozen snapshot. I had to restore my full vmware disk blocks from a backup and then delete the pointers vmware had so that it would not try to restore the image anymore and just boot it from scratch.

Anyway, I can now run my vmware image with either VirtualBox or Vmware Player, although I'm going to stick to Virtualbox from now on: I tested USB on it and it worked fine to talk to my garmin GPS, so that's all I need.

Little things I noticed:

But all in all, while I'm sad to have had to ditch Vmware after about 10 years of being a user, I'm a happy Virtualbox user now. I just hope Oracle isn't going to screw it up.

Just to say that I'm not picking on plymouth, other random things broke during my recent upgrades, and I fixed it (including Xorg switching to kernel modesetting, and requiring i915.modeset=1). Usually it's just a matter of googling for error messages and applying the answers.
(plymouth is a new 'feature' that hides all the boot messages by default, replaces them with a splash screen and tries to capture text output from the boot and log it, which it still does improperly as of today, including by dropping them if there are too many).

A few release backs, the switch to upstart was a bit painful. I can't say I was super thrilled with it, especially when at the time documentation and debugging info was sparse. This has however been fixed (the documentation that is), and while there are still bugglets here and there (statd won't start properly on my laptop), at least my boot doesn't randomly hang on networking dependencies anymore ( https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/499361 ). But the main part is that I'm willing to be more patient and understanding with upstart because it is a clear win for linux: it is good progress on functionality.

And this brings up to the trainwreck otherwise known as plymouth.

So, you might ask, what is so wrong with plymouth? Well, how about this:

• If you are going to be significantly changing all our systems, you'd better post a good rationale as to why the change is good and desirable. This was done for upstart and even mostly for network-manager (nevermind the many problems that network-manager had for 18 months after being pushed to all before it was mostly useable and stable). The cynic will point out that lack of rationale for the switch to plymouth is that said rationale is very dubious.
• If you are going to be pushing a big change to all our system (hell, it's only how they boot, how they fsck drives, and how you can get to single user mode, or not), you'd better bad a page that explains how it works, how you debug it, and how you work around it if you have to. Why is it that even getting 'noplymouth' as a boot option is such a highly guarded secret apparently?
• If you really need to put a multiplexer in the boot system, which is a big change, make baby steps: add it while keeping text mode booting by default for at least existing installs. Debug the hell out of it (it looks to me that as text mode booting as you can get with plymouth is a very little tested code path, unless the entire code is as buggy still as what I experienced in text mode).
• Who thought that they were going to make friends with us by forcing that utterly useless "you don't need to see your boot, it might have useful debug info that might be useful to you, we can't have that". Why is it that is is so fscking hard to turn off plymouth boot message stealing? Even after turning off the graphical crap, you still get a useless one in text mode spinning around colored text dots. Really ?!?!
  noplymouth INIT_VERBOSE=yes at the lilo prompt seems to do it for me right now, but where on earth is that documented?

So, that's really my beef with canonical on this one: I care much much more about having a system I can upgrade mostly safely like I can in debian, than this graphical crap that is downward hurtful to my system. I really don't care about how windows-like you can make linux look like (up to an un-debuggable and opaque boot), I'm really not interested.
Ubuntu/Canonical, stop the madness, please. Do impose some standards on your eager developers who think they came up with the last 'this is so cool' thing to add to linux, especially when it affects essential parts of the system.
I think I'm also specifically bitter about plymouth in ubuntu because its presence could have been made optional in init scripts (Red Hat even had such support in their init scripts), but in ubuntu "it's obviously good enough for everybody, so eat it and shut up".

For more details on what went wrong this time with plymouth, if you are curious:

I could go on (it does go on), but that's long enough, you get the idea....

As sad as it is, and with people like Steve Langasek ( https://bugs.launchpad.net/ubuntu/+source/plymouth/+bug/665789 ) who are now probably understandably defensive about plymouth, likely due to many complaints like mine, instead of doing the right thing and making it really optional, writing a rationale on ubuntu.com as to why it's there and why we should love it, and especially documenting the crap out of it, I'm now dubious as to whether ubuntu/canonical will get a clue about this and whether I should just switch back to Debian where I haven't seen much insanity like this...