Marc's Public Blog - Linuxha

2 way talking to a Lockstate LS-DB-500R lock because the Lockstate wifi Remotelock is no good for me

2013-01-30T16:00:00Z

WARNING: because lockstate is not willing to release anything about the safety of their RF protocol, you should assume that it is vulnerable to both replay attacks (i.e. a neighbour or a van listening to your unlock code, and replaying it later to unlock your door), as well as high speed code sweeps (assume that it is possible to have a custom transmitter transmit all unlock codes in a few seconds and unlock your door).
This does mean that without an official statement from Lockstate on the safety of their protocol, you should not rely on LS-DB-500R to safeguard your door against attackers.

The rest is given for entertainment purposes only:
So, I've been wanting to have a deadbolt (not a lock since electric strikes won't work on my front door), that I can control from my misterhouse PC. For a while smarthome has had an insteon module that talks to Morningstar deadbolts, but it's not a 2 way link: there is no way to know if the door is locked or unlocked.

As a result, I never bought the smarthome/morningstar solution since it was important to me to know that my door wasn't closed right, or closed but unlocked.

Then, I found out about lockstate, and quickly found http://www.lockstateconnect.com/ . A wifi lock, that sounds great, right?

It reports local changes

No wiring required

But, then I read up more about it, and found out it wasn't actually what I was hoping for, nor is it something I can recommend:

It will only talk to their server over the internet. Sorry, but I don't want MY lock to report state and be controllable by a remote company

How well will your lock work if that company is gone in 5 years, or decides to stop supporting their lock (don't think this never happens, it has many times for other companies 5-10 years after the product is shipped).

I want lock updates to go directly to MY server, not theirs.

I want MY server to initiate door unlock right away if needed (that lock will only respond to remote events every minute or so depending on how quickly you want the batteries to die, unless someone pushes a button). This is ok for many uses, but not the 'lock the unlocked door now' required by the bug I'll explain below.

Right, you said lock bug?
Yes. After some testing, I confirmed that there is a flaw in lockstate locks with auto-lock. If auto lock fails just once, because your deadbolt is not aligned with the hole (happens easily on my door if I push it too far or not far enough), the firmware will remember that auto lock is not working, and not try to auto lock later.
The big flaw is that if you manually close the lock, the firmware is not aware of that fact, and it will refuse to autolock until you either power cycle the lock, or your force a lock even using a motor (either using the lock button on the wrong side of the door when you're inside, or using the remote control for the -R models).
I have notified lockstate of that flaw, but they didn't seem very interested in notifying their customers about it. The fix would include having their CPU read back the microswitches state even when it wasn't given a command (probably having a microswitch event trigger an interrupt that wakes up the CPU and tells it to notice the new state and re-enable auto lock if the deadbolt just got locked ok).
I have personally fixed that problem by having my computer keep track of the lock state and try to re-lock it using the RF remote if it stays unlocked for too long. But understand that without that, you'll most likely end up with the door unlocked sooner or later if you don't close it in perfect alignment every single time.

Proof if you don't believe me (it has 30 second delays since I have my door auto locks after 30 seconds)

Ok, so the wifi lock isn't it. What now?
Well, I did go with lockstate afterall, because they did have cheap locks that exactly matched the black color motif of my door, and aesthetics was important for the WAF :)

I bought the Lockstate LS-DB500R-RB (rubbed bronze) because it was a great deal for $100 onsale, and I knew I could very likely hack it to do what I wanted. So, it comes with a remote control, and I knew I could trivially connect its micro switches to a relay and control the door from my PC, just like the Morningstar option, but bypassing insteon and another module I'd have had to pay for.

The more important part however was how to know whether the door was closed, and whether it was locked.

The closed part, I solved first by simply using an X10Sec/X10RF DS10A door/window sensor: http://www.x10.com/security/ds10a_s.html . This is a wireless option working on batteries, and it will fail after 1-2 years when the batteries die, but I already have a fully working monitoring setup with those and misterhouse will tell me when the batteries are low or dead.

Next, I just did not want the lock to require batteries, see amazon comments like this one, so I decided to hot wire the door to a power supply.
First, I used a flat phone cable, which I painted, and only found out later that there wasn't enough copper in that cable (just a few thin wires) and it didn't have enough 'oomph' to actuate the motor, doh! So, I put a second thicker cable with enough copper for just the power, using the phone cable for the return path. I just wired the power where the batteries would go if I had put any.

Yes, the wires fit on my door because I have a small gap. Your door may be different

Return path you said? Yes, I found out how to connect to unused pins on the microswitches and know whether the door is locked or unlocked. You can do this easily without even soldering. See picture:

Be very careful not to pull the gears. They are in a very specific place and you'll be sorry if you have to put them back after you lost which way they went.

Now, if you're counting, you'll say that I had 4 wires in my phone cable, and I only used 2 to get the lock/unlock status. It would be nice if the other 2 wires could be used to tell the lock 'lock' or 'unlock', but AFAIK, this is non trivial to do since you don't want a lock to unlock by just easily shorting 2 wires on the board. This is why I got the LS-DB500R with remote control support.
The interesting bit is that the remote receiver is in the keypad, and the keypad actually will tell the lock CPU: "I received an unlock code, please unlock". This does mean that lock/unlock can be sent over the few wires that go through the door, but I'm somehow hopeful that it's not as easy as just shorting 2 wires :)

So, instead of explaining in long details, I'll show a few pictures on how to modify the remote control to connect it to relays. I used a small breadboard with a voltage regulator to get 3V from 5V which I have on my 1-wire relay board, and you can send lock/unlock codes by shorting one of the green wires with ground.

note the holes for power wires, nicely labelled 3V + -

finished setup with voltage regulator and power LED (since the remote will not show if it has power)

And I then connected everything to my 8 Channel 1-wire I/O board from hobby-boards.com.

If you look at the above picture carefully, relay 7 is an input relay that gets the 2 wires saying whether my door is locked. Locked shows as LED on. Relay 5 is an output relay to send the unlock command to the remote, and relay 6 sends the lock command. From there, you can watch the video showing how sending the unlock command from my PC (off camera) shows relay 5 going on, then input 7 going off showing unlocked, and later relay 6 going on for lock, closing the lock sense relay and turning input 7 LED back on.

And this is how I got a fully locally controllable lock with lock state sense and door open sense for barely over $100 and a little bit of time. See more images for 2 way talking to a Lockstate LS-DB-500R lock because the Lockstate wifi Remotelock is no good for me

Recent Power Monitoring and Misterhouse Talks

2011-11-11T16:00:00Z

For the benefit of my linuxha blog, I should add a couple of links about relevant talks I gave this year.

At linux.conf.au 2011, I gave a talk on Saving Money with Misterhouse: Running Your Lights and HVAC system, and other home automation tricks.
The video can be seen here.

Later, I gave a talk on Power Monitoring with Linux (including how to use the ECM1240 and graph its output with cacti) at Linuxcon 2011.

Weather Monitoring with Oregon Scientific WMR968

2011-08-03T15:00:00Z

A fair amount of people get weather info (wind, rain, and others) using 1-wire equipment, which is a pain both for having to run 1-wire to places outdoors (although I already was doing that), but more specifically because the mast the wind sensor is on is a lightening rod which has fried many people's home computers and networks.

The simplest way to avoid this problem is simply to go wireless so that lightening does not reach your inside computers and networks. As a side note, the WMR968, while far from perfect, is cheaper than 1-wire equivalent solutions (only $200). The equivalent functionality with 1-wire is about $500

The kit comes with solar panels that recharge your device batteries, so they're supposed to run forever (i.e. until they eventually die), and you get:

Indoor temp / humidity / baro pressure

Outdoor temp / humidity

Wind speed / direction

Rain gauge

wind sensor on the roof with the portable receiving console

rain sensor on top and hygro/temp sensor in the shade under the roof

little 'hack' to make the rain sensor more sensitive

The main thing that is nice with that kit outside of the fact that it's cheap, is that you can buy more receivers than there are in the kit (like multiple indoors BTHR receivers to get humidity in different rooms) and while the main console will not receive more than one, the rfxcom receiver I have plugged in my PC can receive as many as you can have (even your neighbours' :) ).
This allows for having the console in the kitchen for instance, and not have to run a cable to your computer in the closet by having it use its own receiver with a bigger antenna and processing ability for many more devices (see list).

Here is a link to all the WMR 968 graphs received from the rfxcom via XPL, with a couple of samples below:

The only caveat is that it's a bit cheaply built, some of the outside sensors are not super water proof, so I slightly modified mine to make sure I had good water seals, and so far so good: can't beat the price and it hasn't failed for me yet :) Not having to climb on the roof to change batteries every so often is a nice bonus :) See more images for Weather Monitoring with Oregon Scientific WMR968

Hacking an external antenna onto an X10 CM19a, and adding misterhouse support

2011-07-24T15:00:00Z

While this page is about the CM19a, the code I wrote should work just as well with the CM15a.

I had an old CM19a (USB X10RF and X10Sec transceiver) lying around. This was more desirable than the well known CM26a used by many misterhouse users in that on top of being USB, it more importantly can decode X10 Security RF signals, as well as send X10RF signals too.
Now, the problem with the CM19a, is that like the CM26a, it has a useless antenna and therefore a useless RF range. The good news however is that the same antenna hack that can be applied to the CM26a works with the CM19a too.

For pictures, see the example for the CM26a antenna modification this hack was based on (scroll to the antenna plug wiring).

In a nutshell, you cut the antenna wire to go far enough to reach the new antenna plug that you attach to the plastic (I used my soldering iron to burn a hole through it). The antenna wire will plug to the center connector. Then, the tricky side is to use the piece of wire you cut off, and connect it to the ground plate of that board. Ground is actually easy to solder to: most holes through the board with metal on each side are ways to pass ground from one side of the board to the other. I used one of them (see red arrow) to solder my other wire to, and connected that to the ground of the new plug.

You then have the option of using a dipole antenna or a quarter plane antenna to connect to your new plug (which one is best depends on where you put the antenna and what kind of area you are trying to cover).

By now, you actually have turned your CM19a into a device that's almost as good as a W800 for reception, but with the bonus of being able to send data too.

Next (for me), was making use of this in misterhouse. Because it is a USB device that does not emulate a serial port, it will not work in misterhouse without a special driver.
To find the simplest way to solve my problem, I decided to use the open source mochad to talk USB to the device and spit out the data frames it was receiving (I hacked mochad to spit out undecoded data). I then wrote a glue shell script that sends that data to a pipe which misterhouse can then read from. It then sends that data into the misterhouse X10_RF module, reusing the common decoding and injection code used by the X10_W800 and X10_MR26 misterhouse modules.
This all ended up in the new lib/X10_CMxx.pm module I wrote and added to misterhouse svn.

You can find more details on the Misterhouse Page for X10Sec and X10RF support with CM19a and CM15a through mochad.

Here are two examples of logs with debugging enabled:

02/07/2011 14:51:45 CMxx: Ignoring first send of X10RF data from mochad (looking for confirmation resend): 8F 80 84 7B F1 80 02/07/2011 14:51:45 CMxx: decoded data received from mochad: 07/02 14:51:45 Rx RFSEC Addr: 8F:F1:80 Func: Contact_normal_min_DS10A 02/07/2011 14:51:45 W800: security: unmatched device 0xf1 (state = NormalMin) 02/07/2011 14:51:45 CMxx: X10RF data from mochad: 8F 80 84 7B F1 80 02/07/2011 14:51:45 X10_CMXX: security: unmatched device 0xf1 (state = NormalMin) 02/07/2011 14:51:45 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 2): 8F 80 84 7B F1 80 02/07/2011 14:51:46 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 8F 80 84 7B F1 80 02/07/2011 14:51:46 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 8F 80 84 7B F1 80

02/07/2011 14:51:59 CMxx: Ignoring first send of X10RF data from mochad (looking for confirmation resend): 60 9F 20 DF 02/07/2011 14:51:59 CMxx: decoded data received from mochad: 07/02 14:51:59 Rx RF HouseUnit: A1 Func: Off 02/07/2011 14:51:59 XA1AK: testx10 off 02/07/2011 14:51:59 CMxx: X10RF data from mochad: 60 9F 20 DF 02/07/2011 14:51:59 XA1AK: testx10 off 02/07/2011 14:51:59 XA1AK: testx10 off 02/07/2011 14:51:59 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 2): 60 9F 20 DF 02/07/2011 14:51:59 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF 02/07/2011 14:51:59 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF 02/07/2011 14:51:59 CMxx: Ignoring duplicate X10RF data from mochad (dupe cnt >= 3): 60 9F 20 DF

Ultimately, mh without debugging only sees, which is what you want: 02/07/2011 14:51:59 XA1AK: testx10 off

1-wire Saved Our Freezer Food

2011-03-11T16:00:00Z

So far, most of my temperature monitoring has been informative but hasn't really saved the world per se. Well, this time I started getting warnings that our garage freezer temperature was getting too high.

Turns out that the freezer door had not been closed right (almost but not right, and it was enough not to make a tight seal). Unfortunately I found out right has we had left the house for a few days, so the freezer had to suffer through with the air leak, which created a lot of frost inside, but I was able to take care of it as soon as we got home, and all was well.

Fine grained house-wide power monitoring with Brultech ECM1240, ecmread.py (with net metering support), and graphing with cacti

2010-08-13T15:00:00Z

Introduction

Until recently, I had a Brand One Powermeter to measure PG&E Meter, PV system and my AC. It was bulky, unreliable, and impossible to reprogram. In other words, it was a poor and expensive solution. That said, I still got some data and reasonable graphs from it as per this earlier blog post.

But let's be honest, I really didn't like that monitor and wanted to ditch it. After some research, the ECM1240 is the best feature/cost ratio power monitoring device I found. You can read about it on the brultech ECM1240 page.

Why is it better than the alternatives?

you can monitor 7 (!) channels plus voltage for less than $200

you can use multiple devices to monitor more than 7 channels (I monitor 20 in my panel)

it comes with multiple CTs to chose from, from highly accurate high current split CTs or TTs to small quarter sized CTs that are appropriate for monitoring all your smaller loads behind each of your circuit breakers

you can monitor let's say 6 circuit breakers as one channel (like 'all lights').

the data gatherer can be connected to your computer via serial port (what I used), ethernet, or wireless (for comparison the TED device, aside from being a single channel device, can only communicate over your power lines, which is unreliable and almost a guaranteed disaster if you use X10 or insteon home automation).

the owner is helpful is responsive to intelligent questions

while the software is meant for windows, data can be gathered on linux or any OS that can run python (i.e. just about anything) thanks to ecmread.py provided in this page.

Here's what it looks like:

the whole system

I calibrated the TTs vs the split 60CTs and the small CT-40s by comparing measurements of the same load

the 2 white boxes are the ECMs1240s, but I also have my older and bigger brand one power meters in there

the small donut CT-40s are great, they take no room at all

After getting this installed, I was able to get data on linux after I got a working but incomplete (for me) ecmread.py from prior authors, Brian Jackson, Kelvin Kakugawa, and Amit Snyderman. I modified it to support net metering and show high precision data as required by proper per second graphing in cacti.

Code

Here is a link to my improved ecmread.py.

And is here my ecmread logfile to cacti/rrdtool converter.

My init script

My script to add labels to each channel

My generic page with logfile to cacti converters.

Gratuitous Graphs :)

Ok, first you can find all the graphs here: all regular owfs derived graphs.
And here are the interesting composite graphs.

So, US houses come with 2 120V phases. Now, if you wanted 100% exact wattage measurements, you'd have to measure the voltage on each and every circuit breaker you measure, but in reality measuring each phase is close enough.
In real life, measuring amps on one phase with voltage from another phase will only give you about a 1-2% error at worst, so it's not a lot to agonize about. In my setup I tried to measure phase 2 loads on my ECM that's plugged into phase 2 power, but wasn't fully able to do it, and it's not meaningful when you measure 240V loads anyway.

This is what the phases look like, as the graph shows phase 1 typically gets more power than phase 2 for me, but depending on the load in my street and my house, they sometimes become close or equal:

Of course, I have a lot of single interesting graphs. Can you tell when my disk to disk backup completed? :)

More importantly, and worryingly, compiling the same kernel took 30mn and 20W on my dual core duo laptop:

While compiling the same kernel on my dual Xeon P4 server took 2H and 80W-ish:

Another interesting graph was charging a 12V marine battery for my UPS:

Thanks to this graph, I was able to find that my TV and speakers took 30W when off. I got a smart power strip that turns them off totally and saved about 30W off my base load:

Ever wonder how much power your fridge is really using?

So, how much power does AC use? Well, not only 3500W for AC, but another 1000W for the whole house fans:

A cool graph showing House Power Use (calculated) from PG&E meter and PV production probe:

And for the money shot, all the house uses combined on one graph:

And the same graph, but with AC that was activated:

Setting up cacti

See my cacti config page.

Now, the tricky part is creating graph items that do not exist (like house use, or unmonitored house use). See this post I made on how to do this.

The other tricky part is that I had my ECM graphs setup to refresh every 10 seconds, which is faster than the cacti poller which runs every minute. This post explains how I did a faster than 1mn refresh in cacti See more images for Fine grained house-wide power monitoring with Brultech ECM1240, ecmread.py (with net metering support), and graphing with cacti

Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti

2010-08-06T15:00:00Z

Introduction

In a prior post, I wrote about using digitemp to talk to 1-wire temp sensors.
Digitemp is a good first choice if you only care about temperature since it's already available in most linux distributions, and it's pretty easy to setup. However, digitemp is fairly limited: first it does not support anything but temperature, and one humidity sensor (with only one convertion table, and there are several depending on which one you end up buying). On top of that, the humidity sensor only works with the serial interface and not the USB one.

1-wire supports a lot more than just temperature, including several kinds of humidity sensors, a moisture sensor which can be used to measure the amount of watering needed for your lawn, and outdoors UV and solar intensity sensors (available from hobby boards).
1-wire also has weather stations, but you do have to worry about your pole with the wind sensor being hit by lightening that would then be channeled back inside your house and to your computer (not good). Also, I have not found 1-wire weather stations to be price competitive with an Oregon Scientific WMR968 which is wireless and can be directly connected and read from via its serial port on linux and through misterhouse.

http://www.hobby-boards.com/catalog/index.php?cPath=22

wind: $140 (95+45) (or some more expensive $225)

baro: $60

humidity/solar x2 (indoors/outdoors): $120

rain gauge: $93

then add wiring and a possible hub ($50)

This gives you a total of $493. For comparison an oregon scientific WMR968 weather station costs $200 (see this thread for details on that) and my WMR 968/rfxcom page for my setup.

After switching from digitemp, I realized that my single bus described here was really stretching the limits of a single bus. I therefore ended up adding a hub.

What you need to know about the hub is that you get 6 buses split off your original bus (i.e. you get 7, one is the original bus passed through but with bus power added, and there are 3 1-wire switch chips with a MAIN and AUX sub bus each, effectively giving you 6 more busses). The Hobby-Boards 1-wire hub, which is the one I bought, injects regulated 5V power and unregulated 12-24V power for some special outdoors 1-wire devices on all the ports, including the pass-through one.
Having a powered bus with 5 and 12-24V allows the use of some special 1-wire devices like the moisture meter, and the UV sensor.
If you go with the hobby-boards hub, you should plan to wire like they do to make your life easier: hobby boards wiring chart.
.

1-wire basic setup

I talked about what to buy, and how to wire on my last Temperature monitoring and graphing with 1wire devices, digitemp, misterhouse, and cacti post. Go read that section.

1-wire and star technology

If you have a wiring closet where all your cables come back, you have a star topology. With a hub, you can have up to 7 branches, which may be enough for you. If however you need more than that, when you use the wiring chart from hobby-boards, I recommend that you use pair 3-6 as a return for pair 4-5: i.e. you send signal down your cat-5 to some location and then you connect 4 to 3 and 5 to 6 at the end point. You then connect 3 and 6 to 4 and 5 of your next Cat-5 run elsewhere in your house.
When combining this techique and the 7 branches you get from a hub, this should give you more than plenty branches from your network closet.

Hobby Boards 1-wire UV meter

I added the Hobby-Boards UV sensor to my roof 1-wire bus now that it had bus power added to it after I got the hub, and I used cella-wrap to make sure water wouldn't get in the box or touch the 1-wire device wires and create a temporary bus short. The UV board also has a built in temp sensor, but the weatherproof box it comes with will act as a greenhouse somewhat, so the temperature read by that sensor will be higher than normal (it is used for UV readout temp correction). If you don't care about UV index as much and would like outside humidity level, you may want to consider the humidity/temp/solar sensor instead (I get outside humidity through my WMR968 Oregon Scientific Weather Station, so I didn't care to have that in my outside 1-wire sensor).

UV sensor in mostly water proof box

gets inside the attic through a water proof hole

Hobby Boards 1-wire Moisture Meter

The moisture meter has a temp sensor on the moisture meter control board, but it is unfortunately not used to correct the moisture readings from the moisture probe (this sensor does have a moisture value that can vary up to 50% when fully wet between temperature extremes). Thankfully in real life, your soil temperature deeper down should not change too much and affect the reading too much.
I then buried the control board too so that it could be used as a soil temperature sensor.

moisture meter

the trick was not to bury the sensor too deep: just as deep as my grass roots

protecting the board from water

regular DS18B20 outside temperature probe in the shade

The temperature graph is not super exciting, but still good info. It's reassuring that the dirt temp doesn't change much and therefore the moisture readings won't be foiled much:

The moisture readings are more useful and for instance this graph shows how I wasn't watering enough, and turned up the sprinkler times a little bit so that the humidity ends up at a reasonable state:

But the really interesting graph is seeing soil humidity over soil temperature (green) and outside temperature (red):

what this graph shows is outside temperature obviously affecting soil temperature a little bit, and when soil temperature goes up on a wet sensor, it brings up the moisture reading somewhat unfortunately.
Similarly, from the 1-wire side, you don't see a humidity percentage, but a negative current value

cat /etc/owfs/moisture/35\ Front_Lawn/current; echo
    -1.35875

I looked at the range of values I typically get for full dry and full wet, and this is definitely sensor and board dependent. There is also the problem that the hobby-board design up to 2010/05, did not have a ground loop isolation transformer (without which the sensor would behave erratically as soon as you put it in the ground). Once you add the isolation tranformer, it does change the current values for dry and wet though, so each person may have to compute their own. Unfortunately I've seen my sensor briefly return values all the way to -2.55 instead of the typical wet -1.74
See the owfs moisture sensor page I wrote for details on this.

This is the code I use for now:

# 0.56625/0.5675 bone dry, but once got 0.335625.
# glass of water yielded 1.74 but I've sometimes seen 2.555. What to use?
my ($min, $max) = (0.33, 2.6);

# value should be temp ajusted in an ideal world, but I don't have correction tables.
$value *= -1;
$value = $min if ($value < $min);
$value = $max if ($value > $max);
$value = (100 * ($value - $min) / ($max - $min));

I've struggled on finding the upper value. First, I got -1.74 but eventually I sometimes got -2.55 out of the blue for a few samples, which yielded some clipping, so I changed the max value from -1.74 to -2.6 at noon on this graph, explaining the sudden dip. You can however you can see on the left the jump to 100% when the sprinklers were started:

Despite the sensor being noisy, temperature dependent, and hard to calibrate, the overall graph is still useful enough to show whether your sprinklers are keeping the dirt moist enough, or if your overall moist is just going down (bad).

Owfs setup

But back to 1-wire and owfs, owfs is more work to setup but it's just the way to go if you want anything more than temperature monitoring.
You need to start by getting the latest source code from http://www.owfs.org/, and compiling it. /configure --enable-debian did the right thing for me, but you need to make sure you have the right headers on your system

I wrote this script, read_owfs that reads from an owfs symlink tree and generates a digitemp looking logfile so that it's easy to parse the logfile later with the same code regardless of whether you use digitemp or owfs to capture the data.

owfs with a hub is a bit "interesting" since you have to find your devices in subtrees, which is why I made a symlink tree to make my life easier. I setup a symlink for each hub port and a chain name to point to each of the 3 buses' AUX or MAIN branch. This is what it looks like:

/etc/owfs/bus1 -> /owfs/1F.F05005000000 /etc/owfs/bus2 -> /owfs/1F.E25005000000 /etc/owfs/bus3 -> /owfs/1F.E15005000000 /etc/owfs/chain1 -> bus2/main /etc/owfs/crawlspace_chain -> bus2/aux /etc/owfs/dining_chain -> bus3/main /etc/owfs/roof_chain -> bus1/aux

/etc/owfs/humidity/56 Hall_Closet -> ../chain1/26.2E4DF5000000/HIH4000 /etc/owfs/moisture/35 Front_Lawn -> ../crawlspace_chain/30.131A62120000

/etc/owfs/temperature/11 Family_Room -> ../chain1/10.A8D1ED010800 /etc/owfs/temperature/12 Living_Room -> ../chain1/10.52D1ED010800 /etc/owfs/temperature/15 Garage -> ../chain1/10.2223EF010800 /etc/owfs/temperature/21 Attic -> ../roof_chain/10.5DE1ED010800 /etc/owfs/temperature/22 Roof -> ../roof_chain/10.94A2ED010800 /etc/owfs/temperature/23 Outdoors_Roof -> ../roof_chain/28.57B659020000 /etc/owfs/temperature/25 Roof_UV -> ../roof_chain/EE.E749CB010800 /etc/owfs/temperature/31 Crawlspace -> ../crawlspace_chain/10.F9F3EE010800 /etc/owfs/temperature/32 Outdoors_Crawlspace -> ../crawlspace_chain/10.D1D0ED010800 /etc/owfs/temperature/35 Front_Lawn -> ../crawlspace_chain/30.131A62120000 (...)

This is how I start owfs and the xpl-owfs gateway:

gargamel:/etc/owfs# cat /etc/init.d/owfs 
#!/bin/sh
case "$1" in
  start)
        umount /owfs 2>/dev/null
	# serial
        /opt/owfs/bin/owserver -d /dev/DS9097U -F --error_print 0 --error_level 1 --nozero
	# usb
        #/opt/owfs/bin/owserver -u -F --error_print 0 --error_level 1 --nozero
        /opt/owfs/bin/owfs    -F -s localhost:4304 /owfs --nozero
        /opt/owfs/bin/owhttpd -F -s localhost:4304 -p 8082 --nozero
	# this comes from xpl-perl, reads /owfs devices and relays their data periodically
	# over XPL, where they can be relayed to misterhouse.
        xpl-owfs --interface eth1 --owfs-mount /owfs >/dev/null &
        ;;
  stop)
        pkill -f '/opt/owfs/bin/owfs'
        pkill -f '/opt/owfs/bin/owserver'
        pkill -f '/opt/owfs/bin/owhttp'
        pkill -f 'xpl-owfs'
        ;;
  restart|force-reload)
        $0 stop
        $0 start
        ;;
  *)
        echo "Usage: owfs {start|stop|restart|force-reload}" >&2
        exit 1
        ;;
esac
exit 0

Gatewaying 1-wire data to misterhouse

owfs-xpl is a good way to relay 1-wire data to misterhouse via its XPL gateway as opposed to reading owfs directly since you're assured not to hang.
This is how you setup devices in misterhouse:

XPL_SENSOR, bnz-owfs.*:10.2223EF010800, garage_temp, , temp
XPL_SENSOR, bnz-owfs.*:28.3359C7010000, freezer_temp, , temp
XPL_SENSOR, bnz-owfs.*:28.998D4D020000, computer_closet_temp, , temp
XPL_SENSOR, bnz-owfs.*:26.2E4DF5000000, hall_closet_temp, , temp
XPL_SENSOR, bnz-owfs.*:26.2E4DF5000000.1, hall_closet_humidity, , humidity

If you use xpl-owfs, adding .1 behind a humidity value lets it pick another DA converter between the default one in owfs: .1 for HIH4000, and .2 for HTM1735 (it makes a difference since the analog value is turned into a different moisture percentage value as a result).
Here's what it looks like and the patch for xpl-owfs if you are not running a recent one with that support:

gargamel:~# grep . /owfs/26.2E4DF5000000/{.,*}/humidity
/owfs/26.2E4DF5000000/./humidity:     58.8895
/owfs/26.2E4DF5000000/HIH4000/humidity:     60.5681
/owfs/26.2E4DF5000000/HTM1735/humidity:     56.8768

--- /usr/share/perl5/xPL/Dock/Owfs.pm.orig      2010-04-17 09:08:43.000000000 -0700
+++ /usr/share/perl5/xPL/Dock/Owfs.pm   2010-04-17 09:25:25.000000000 -0700
@@ -154,6 +154,8 @@
   foreach my $dev (@$devices) {
     foreach my $rec ([ "temperature", "temp" ],
                      [ 'humidity', 'humidity' ],
+                     [ 'HIH4000/humidity', 'humidity', 1 ],
+                     [ 'HTM1735/humidity', 'humidity', 2 ],
                      [ 'counters.A', 'count', 0 ],
                      [ 'counters.B', 'count', 1 ],
                      [ 'current', 'current' ]) {

Once you have misterhouse configured to receive XPL messages, with those in mh.private.ini:

ipaddress_xpl_broadcast = 192.168.205.255
ipaddress_xpl = 192.168.205.3
# this is how you turn it off
#xpl_disable = 1
# if you are using a better XPL hub, turn off the mh built in one
#xpl_nohub = 1

Then, you can simply read owfs values in misterhouse via XPL like so:

my $compcloset_temp = $computer_closet_temp->state()

(there is one caveat there: misterhouse does not currently have a timer to remove the temperature value if you aren't getting updates from xpl/owfs)

Note that there are other ways to read owfs in misterhouse, namely:

via xAP (just like xPL, but unless you're already using xAP for something else, don't bother with it)

iButton code in MH if you use the iButton 1-wire interface as opposed to a DS9097U (serial) or DS2490 (USB)

Owfs_item code in MH from Jim Duda. This one talks to owfs via its owfs_server daemon and perl bindings, but I've found this to be had to get to work because the owfs perl bindings haven't been reliable for me. Jim actually recommends using xPL for new installs, but if you want to try his code look at thread #1 (you need to open the messages after this one) and thread #2.

The summary of the threads I posted however is that xPL and xpl-owfs are the way to go for reading owfs sensors in mh.

Feeding data in cacti / code

Now that you have data available in /etc/owfs/*, and being sent to misterhouse via xPL, you can (and should) log it to a file.

I have written two scripts for this:

read_owfs reads owfs data from a symlink tree in /etc/owfs/ and populates a digitemp looking file in /var/log/temperatures.

cacti_owfs reads a digitemp or owfs derived /var/log/temperatures file and converts it into rrdtool data or cacti compatible query data.

The second script is the important one which can generate cacti data or help you regenerate/build an rrdtool RRA file from scratch if you add fields or change your data format (my recommendation is to plan ahead and add extra fields in your RRD file for probes you might add later (you can't add fields after the fact without regenerating the entire file and refilling it with all the data from time 0, so it's better to plan ahead and get it right the first time, and a good way to do that is to just add extra fields that you're not using yet but can use later) ).

For more details on cacti integration, see my Gatewaying 1-wire, XPL (Oregon Scientify Weather), Brultech ECM1240 Power Data, and Brand OneMeter Data to cacti page.

cacti_owfs can also be used for feeding data in an rrdtool after the fact (--dump-cacti) and used like so:

sort -u < dump | time xargs rrdtool update $RRD --template `cacti_owfs --cacti-dump-header`

(after having freshly re-created the RRD and make sure you use --start 1271572300 with the right second value that's just before the first one in your dump).

Setting up cacti

See my cacti config page.

Gratuitous Graphs :)

Ok, first you can find all the graphs here: all regular owfs derived graphs.

And here are the interesting composite graphs.

I keep track of the humidity in our wine closet. This chart shows when I refilled the evaporation water plate:

AC run on a warm day:

I was curious to know if our old fridge in the garage was having unreasonable and too frequent on/off cycles. The old garage fridge does run a bit warmer but doesn't cycle that much more often than the new one, so I'm not as worried about it dying soon as much (it does take a fair amount of extra power though, being 10 years older):

Interestingly enough, it is actually the newer kitchen freezer which has occasional big temp jumps to >50F.

And this lets me keep track of temperature and humidity in our wine closet (the good news is the water plate in the closet does keep humidity higher compared to the house humidity level):
See more images for Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti

Booster Fans and Heating/Cooling with Outside Air with Misterhouse

2010-08-06T15:00:00Z

Introduction

So, my plan was to be able to use outside air to cool the house at night during the summer without leaving doors and windows open on one hand, while being able to boost or shutoff air to the master bedroom depending on the time and boost HVAC air to the family room which gets the warmest but is also the hardest to cool since it's at the end of the line.

Boosting air is simply done by adding booster fans to the right ducts, but controlling the fans at the right time is where it gets interesting: I wanted to be able to decide not to turn on the booster fan to the bedroom during the day (waste of air) and not bother with trying to heat/cool the family room at night either since the master bedroom needs the air more than the family room.
The astute reader will note that this is usually done with dampers, but the furnace we have is older and would require a pressure return for extra air if only one room were targetted, and dampers just add more money and complexity when I already wanted booster fans. The trick is that booster fans can act as semi dampers when they're not running, which is good enough.

Now, the main goal was however to be able to pull air from the outside and bring it in the house. This was accomplished with the same 2 booster fans to create suction this time and 2 dampers do redirect the air from the furnace output to ducts that go outside the house.

that was a few parts :)

damper closed

damper opened (notice the 2 small wires which got to my 1-wire 8 channel IO board)

So, to cool down the house (or bring warm day air in during the day in the winter), I need to:

close the air coming from the furnace with a damper

open the air coming from outside with another damper (both happen at the same time)

turn on one or two of the booster fans to suck air from outside

make sure that at no time my HVAC system actually decides to turn on too, and if it does reset the dampers to allow air from the furnace (which is also the AC).

automatically do the above when there is a suitable temperature difference between inside and outside.

For normal operation, I have 2 HAI RC-80 and RC-90 thermostats, one for the house and one for the master bedroom. Depending on which one triggers, the booster fan to the appropriate zone should activate too.

Do you have a zone board that can do all this?
If so, how many hundred dollars is it and do you need to call an HVAC person with proprietary software to reprogram it?
Do the inside and outside temperature probes cost $3 each, or more? :)

If you want to skip ahead, this is the misterhouse hvac code I was able to write to control the hvac system.

Implementation with Misterhouse

This is where misterhouse comes in again, I can program absolutely any logic I want and mostly with sensors and actuators that cost a lot less than typical overpriced HVAC parts.
The bits that came in play were:

HAI RC-80 thermostat which I can query for status (cooling/heating/HVAC fan on or off?) from misterhouse thanks to the code I wrote earlier (see my misterhouse HAI code). Actually I also use that code to know the temperature in the bedroom (which I could also get with a 1-wire sensor).

1-wire temp sensors inside and outside the house. This is obviously useful to know whether air outside is the right temperature for bringing in, or not (see the Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti for details on the owfs setup.

1-wire 8 Channel I/O board which I use to control 24VAC to the dampers. One is normally closed and the other one normally opened. They each only need about 250mA to be held in the other position, so a single 500mA 24VAC power supply going through a relay on the 8 Channel I/O board is good enough. See my owfs 8 channel I/O page

2 X10 (or Insteon) remotely controlled plugs to turn the booster fans on or off.

Here's a picture of the 1-wire 8 Channel IO board

I then added some 1-wire temp sensors inside the ducts to confirm the temperature of the air going through them, and that's it. All that was left was some code, which you can find below.

When both duct fans are running, the power graph shows they use 300W:

Here is a temperature graph of cooling with outside air without using the AC showing a 5 degree decrease for dining room/family room:

and for comparison cooling with just the windows open, which I don't quite want to do anyway (compare the family and diving room temps which only go down by 2 and 1 degree).

For yet another comparison, the following link is a test with only the family room duct fan running (and not the bedroom one): that was a 3 and 1 degree decrease for the dining room/family room. Not as impressive.

Yes, booster fans do require some electricity to run (150W a piece or so), but for comparison, AC is 3000W just for the AC, to which you have to add 1000W for for furnace whole house fan and then the 150W for the booster fan:

At least, as a bonus, we do now get more cold air to the family room with the booster fan when we do run the AC, so we reach our target temperature more quickly.

Top/Hvac Damper Open to Close Video (click me)

Scripts/code

This is the small script I use to turn the dampers on and off via the 8 Channel IO board.

And here is my misterhouse hvac script that controls the booster fans and outside air intake depending on inside, outside temperature and HAI thermostat status.

See more images for Booster Fans and Heating/Cooling with Outside Air with Misterhouse

Rfxcom, W800, and MR26a on misterhouse, X10Sec Garage and Mailbox Setup

2010-06-25T15:00:00Z

I spent a long time reading and writing documentation on how to use those RF interfaces with misterhouse, and how to use XPL in general.

Instead of pasting the information here, here is the link with all the goods: http://misterhouse.wikispaces.com/X10RFX10SecRFXCOM

I did also learn about RF and antenna design in the process. Here is the quarter wave antenna I built and that I'm now using on my MR26a:

And the simpler, but actually more effective dipole antenna I made and that I use for the W800RF32:

The serial W800RF32 is easier to extend with a long serial cable

One of my first applications was to read an X10Sec module in my mailbox to know when said mailbox was opened:

The next one was to know when my garage doors were opened or closed.
The first thing was to move the door sensors to be at the bumper level, which included drilling holes in the gardrail, no small feat without diamond tip drills which I didn't have. After several hours of making new holes and adjusting the sensors, they are now set to detect the car's bumper as opposed to looking for the car's tires which made it easy to close the door on top of the hood ...

Next, I installed X10Sec sensors on each door so that I know when they get opened or closed.

Technical notes:

X10Sec modules send their code 4 times, but if you miss it because the RF band was noisy or the antenna a bit out of reach (even though I tried hard with a dipole antenna put in the attic), then you have no way of querying the state.

X10Sec modules are however nice enough to rebroadcast their state every 4 hours or so in case you missed the previous state update.

Here is the code I wrote for misterhouse:


#08/05/2010 11:34:22  State Garage2 is alertbattlowmin
#08/05/2010 11:34:22  State Garage2 is normalbattlowmin

my %x10rfstate; #noloop

foreach my $dev (["x10sec_garage1", "Main Garage Door"], ["x10sec_garage2", "Small Garage Door"], ["x10sec_mailbox", "Mailbox"])
{   
    my $obj = $dev->[0];
    my $name = $dev->[1];

    $obj = &::get_object_by_name($obj);
    $x10rfstate{$obj} = $obj->state if (not defined $x10rfstate{$obj});

    if ($obj->state_now)
    {   
        my $oldstate = $x10rfstate{$obj};
        my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime();
        $year += 1900;
        $mon++;

        print_log "X10SEC: $name is ".$obj->state." (was $oldstate)";

        open(LOG, ">>/var/log/x10sec");
        printf LOG ("%.4d/%.2d/%.2d %.2d:%.2d:%.2d $name: ".$obj->state." (was $oldstate)\n", $year,$mon,$mday,$hour,$min,$sec);
        close(LOG);

        if ($obj->state ne $oldstate)
        {   
            if ($obj->state =~ /alert/)
            {   
                system('date | Mail -s "'.$name.' Just Opened" EMAIL');
            }
            elsif  ($obj->state =~ /normal/)
            {   
                system('date | Mail -s "'.$name.' Just Closed" EMAIL');
            }
            else
            {   
                system('date | Mail -s "'.$name.' UNKNOWN STATE" EMAIL');
            }
            $x10rfstate{$obj} = $obj->state;
        }
    }
}

# vim:sts=4:sw=4

See more images for Rfxcom, W800, and MR26a on misterhouse, X10Sec Garage and Mailbox Setup

Powering UPSes with Marine or AGM batteries

2010-06-18T15:00:00Z

So, isn't it a bit ironic that if I have solar panels that power my entire house and then some during the day, if PG&E power goes off, my inverter has no choice but to shut off too, likely both because it uses PG&E as a phase offset source, and because it would likely be bad to feed power on a down PG&E line (or at least would make it harder for PG&E to diagnose, not counting the fact that the power grid acts as a regulator in case there is too much or not enough power).

Anyway, I wanted to have power a bit longer than 10mn when PG&E power went out, so I looked into UPS solutions while powering from a Marine or a AGM (Absorbed Glass Material) battery, both being better options than a car battery (car batteries are not meant to be deep cycled or use for extended periods of time and they do put out hydrogen when you charge them, which is undesireable inside a house).

Interesting bits I found out:

This page http://www.type2.com/library/electrip/battbas.htm has decent battery info, and this one gives a bit more about AGM batteries http://www.windsun.com/Batteries/Battery_FAQ.htm . Yet, even marine or AGM batteries are not meant to be discharged at high rate. If you do, they may not even put out half their rated capacity.

UPSes aren't meant to be used as inverters for long time power off situations. Some will actually use more than one battery (increases voltage, decreases amps, which is good for lowering draw, but makes it a pain when you need multiple big matching marine batteries).

Drawing 1400VA from an APC UPS at 24V is about 60A! That's about 3-4 times what the average copper wire going to your house outlet is supposed to carry. These pages: http://en.wikipedia.org/wiki/American_wire_gauge and http://www.powerstream.com/Wire_Size.htm give some guidelines, but long story short, you need a very thick wire to carry that many amps over more than 10cm without heating up your wires (never good) and wasting a lot of power through resistive loss.
I first quadrupled up my initial wire but then realized that it only took me from 18 gauge to 12 gauge, which was a bit low, so doubled that up with a cable I cut off from a jumpstart cable (that one was plenty thick, it was more an issue to solder it to the rest due to how massive it was and how underpowered my soldering iron was for the job).

In the end, I got a couple of mostly matching AGM batteries and went ahead with those for my 24V APC 1400 SmartUPS. I know my runtime wouldn't be fantastic but good enough considering (at least until I can find another free marine battery :) ).

At 400W of actual use, that was 40% of the load for the APC 1400 inverter and a self test estimated that I would get about 1H of runtime, which gave 16Ah usable for my AGM batteries instead of about 55Ah at a lower draw (when new, and they were not new). It's a bit disappointing on one side, but on the other side, 1H of runtime for my computer gear ain't too bad, especially for AGM batteries that cost $20 a pieces (cheaper than the stock 7Ah batteries).

first I quadrupled the wire.

this is a big ass gauge jumper cable

what it looks like connected

Anyway, even if this isn't quite the runtime I was hoping for, and it seems hard to find any UPS with a good VA rating by using a single 12V battery (not too surprising due to the amp problem I pointed out), this is still pretty cool. See more images for Powering UPSes with Marine or AGM batteries

Dealing with many USB to Serial Port Converters on linux and device naming

2010-06-17T15:00:00Z

So, if you have more than one USB to serial adapter, you might have issues with which order they load in, causing the serial port numbers (/dev/ttyUSB0 .. x) to change pseudo randomly at boot, it's pretty annoying.

In my case, I picked up an 8 port ftdi usb to serial converter for only $15 on Ebay (woot!) but even its ports may not come up in sequence at boot time if other modules are loaded at the same time (a race condition can cause the pl2303 port to be in between one of the 8 ports from the ftdi hub).

Turns out there is a solution to this problem with udev, iff your usb to serial adapters have serial numbers (my ftdi ones did, but my pl2303 ones did not), or if you only have one serial adapter of each brand (like ftdi, pl2303, keyspan).

In my case, I have (don't laugh), 11 serial devices on my server:

Server serial console

1-wire serial adapter

Insteon PLM

APC ups

W800 X10RF and X10sec gateway

X10 MR26a X10RF gateway

First ECM 1240 power meter

Second ECM 1240 power meter

Brand One Meter serial adapter

HAI thermostats (2 on one line)

Rfxcom 433.92Mhz (433.92Mhz RF gateway for Oregon Scientific Weather Sensors)

This is what you do with udev:

gargamel:~# cat /etc/udev/rules.d/50-local-usb-serial.rules 
# /etc/udev/rules.d/50-local-usb-serial.rules

# http://www.reactivated.net/writing_udev_rules.html#udevinfo
# was udevinfo -a -p /class/tty/ttyUSB0
# now udevadm info --attribute-walk -p /class/tty/ttyUSB0

SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvN", SYMLINK+="usb-serial-ftdi-8_1"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvO", SYMLINK+="usb-serial-ftdi-8_2"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvP", SYMLINK+="usb-serial-ftdi-8_3"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvR", SYMLINK+="usb-serial-ftdi-8_4"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvQ", SYMLINK+="usb-serial-ftdi-8_5"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvS", SYMLINK+="usb-serial-ftdi-8_6"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvT", SYMLINK+="usb-serial-ftdi-8_7"
SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A7006gvU", SYMLINK+="usb-serial-ftdi-8_8"

SUBSYSTEMS=="usb", ATTRS{product}=="FT232R USB UART", ATTRS{serial}=="A800dMFk", SYMLINK+="rfxcom"

SUBSYSTEMS=="usb", ATTRS{product}=="USB-Serial Controller", ATTRS{manufacturer}=="Prolific Technology Inc.", SYMLINK+="usb-serial-pl2303"

After making a change udevadm trigger will update symlinks.

One Year Of Solar Panels

2010-03-11T16:00:00Z

Today is the one year anniversary of the Solar Panels Turnup, so I thought I'd gather a few stats to see how they did (unfortunately the full house monitoring system only became active more than 2 months later, so I don't have full stats).

Cobalt Power Estimated the panels yearly production at 8114Kwh for a year. I was a bit apprehensive during the quotes because of our weirdly shaped roof and shading issues from nearby trees, but in the end we got exactly 8700Kwh a year later, or almost a 600Kwh production beyond the estimate. Great!

Cobalt Power estimated our federal tax credit to $11,235 (the CA tax rebate is directly taken off the initial bill). I just got my taxes done and our tax credit was $11,746. this made the panels around $500 cheaper than quoted. Great again! :)

According to PG&E, we used around 1600Kwh since last year (i.e. we used about 1.5 months' worth of electricity), but due to the time of use credits (summer daytime electricity that we mostly produce is worth more than night and winter electricity), we actually got a $40 credit for the year, so we won't owe them anything outside of the monthly connection charge ($10, which is much cheaper than batteries and being off-grid). This year the $40 is lost but next year, they'll actually owe us whatever credit we get (i.e. we get a check back).

At this rate, we should break even in 5 years.

So in a nutshell, the system performed great so far. Cobalt Power definitely underpromised and over delivered and we're apparently done paying electricity bills from now on.

Here are the not quite yearly graphs (as a reminder you can zoom in by selecting a rectangle with the first mouse button and zoom out with the second button).

As a slight repost, below is a graph with production start and end (hour the sun starts hitting the panels and when it sets past the roof). Note the one hour jump on Nov 1st because of the daily savings change.

Production hours:

Production Kwh (nice to see that March is getting a lot more sun):

Last, but not least this page lets you get all the daily starts (note, you need to change the top preset to 3, 6, or 12 months to get useful data, last day doesn't show anything).

Daily Power and Solar Stats

2010-02-14T16:00:00Z

It's not quite a year yet, but I've recently worked on daily stats for our solar panels. If you go to the Solar and Power Monitoring post, you get all the stats that are gathered every few minutes.
I was however missing daily stats showing for each day what time production started, what time it stopped (depending on dailight savings time of course), how many hours of sun we got. I also added a repeat of how many Kwh were produced that day.

Here are the resulting graphs (as a reminder you can zoom in by selecting a rectangle with the first mouse button and zoom out with the second button).

The good news from that data is that even the worst day, we made at least 3Kwh, which is what some small apartments use in a day. The average at the end of a year cycle should be around 24Kwh/day, which is 80% of what we end up using and less that what we'll end up paying once you figure out time of use offsets (i.e. the average electricity we make is worth more than the electricity we use).

Below is a graph with production start and end (hour the sun starts hitting the panels and when it sets past the roof). Note the one hour jump on Nov 1st because of the daily savings change.

Production hours:

Production Kwh:

Last, but not least this page lets you get all the daily starts (note, you need to change the top preset to 3, 6, or 12 months to get useful data, last day doesn't show anything).

Temperature monitoring and graphing with 1wire devices, digitemp, misterhouse, and cacti

2009-12-23T16:00:00Z

If you are interested in this page, you should also refer to my later Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti blog post.

Temperature logging

I had basic temperature logging via the HAI RC-80 thermostat code I co-maintain which polls our thermostats for temperature (along with other settings like whether they are running). Other info on the RC-80 thermostat can be found here, just note that they cost about $50 used on Ebay.
Our house is a little different from the average: we have a sub-zone in the master bedroom which is basically a second big vent going directly from the HVAC system to the bedroom. The bedroom has an RC-80 which turns on the HVAC system and opens the bypass vent when the master bedroom temperature needs to be adjusted, however there is still air going to the rest of the house too, but since a lot goes to the bedroom, it gets to temperature more quickly. So you may ask why the subzone is not just a separate zone and that's because our HVAC system is too old, cannot be run a less than full power and would get damaged if its output was only sent to our bedroom.

The RC-90 looks pretty simple

the interesting part is the added wires for the serial port

The main zone is controlled by an RC-90 (which I use as a regular RC-80). When it turns on, the entire house gets air, including the master bedroom through its smaller vents.
The little trick is that the zone board (which gets input from both the RC-90 and RC-80) is configured to run both zones for 2 minutes after the heating or cooling has stopped (probably to recirculate the air, even in the unused zone).

HVAC zone board

The one nice thing is that the HAI thermostats tell me what they are telling the hvac zone board, so I can tell when the command was sent, for how long and compare with the HVAC output with temp sensors. The thing that is not as good is that the temperature I get back from the HAI thermostats does not have any decimals (as much as I hate Fahrenheit, I log temps in Fahrenheit because with integer only temperatures I get somewhat better resolution in Fahrenheit than in Celsius).

Here's a good example graph showing combined data:

The graph shows the bedroom HVAC being activated (orange) with the resulting rise of both temperature probes since the subzone has to activate the main zone too). When the main zone is turned on (red), the subzone vent only shows a slight rise in temperature due to leakage in the valve, except in the last 2 minutes right after the HVAC system is turned off but the fan left on: you can see the vent being opened right after the HVAC is turned off.

You can compare with the simpler graph based just off data from the HAI RC-90 and RC-80 and data gathered from the misterhouse HAI code I improved. The following graph shows AC being turned on for the main house and the master bedroom after that:

Since the 1°F resolution of the HAIs is not good enough for nice graphs, this is where the 1-wire DS18B20 temp sensors come in. In most rooms I just have them plugged into the wall via Cat-5 (more details below) and in order to save having to make another hole in our ducts, this is how I measure the main duct temperatures when air is being blown :)

Findings

So that's for the HVAC system but after that I was curious to know:

how balanced or misbalanced the hvac system was (which rooms get warmer or colder). In our case, we're actually happy to have a misbalance as we don't need to heat up or cool down all the house evenly. The following graph is telling, it shows that using the main HVAC system gets our main corridor (where the thermostat temp is sampled) warmer than than our master bedroom with the family room being warmer than our living room (which is good since we spend more time in the family room than the living room). Of course, it also shows our family room has more leakage, likely through the fireplace vent. The good news is that the wine closet is well insulated and shows a relatively constant temperature, so I am hopeful that we may be able to hack in an insteon controlled AC system that exhausts to the attic and cools down the wine closet (the only issue is controlling the humidity since an AC system will get the temperature to a nice value, but also dry up the air).

the temperature under the house in the crawl space to know if we can get air from it to cool down the house more cheaply during the summer by grabbing air from under the house during the night. The first graph shows temperatures during the winter on sunless days (roof shingles temps otherwise still get over 90 degrees in plain sun during the winter). The second graph during warmer end of summer days shows that we may be able to get cool air from under the house instead of running the AC (will need HVAC modifications).

If you wonder why the attic and garage lines are squiggly, that's because of the computer closet exhaust fan to the attic and the heat from the fridge compressor in the garage.

how warm the roof and attic get during the summer is also interesting (the above graphs both show how warm the roof shingles get, how it wouldn't be hard to heat water there, and how attic temperature closely follows the roof temperature even with the exhaust fans we have in the attic)

outside air temperature at the top of the house vs at its base in the shade. It make sense that the air by the crawlspace is less variant and more stale, keeping a more constant temperature, while the air by the roof (sampled under the roof to negate roof sun heating) moves around more with wind.

the temperature in our lesser insulated garage and whether that's influencing temperatures in our family room on the other side of the wall, or not. One night, we kind of forgot to close the garage door. This zoom on the Garage vs Family Room vs Outside temperatures graph is relatively good news: the garage temperatures dipped a lot, but thankfully the family room on the other side of the insulated door, did pretty well.

for fun, I looked at how hot my computer closet is getting, and used that temperature to turn the exhaust fan on and off via insteon and X10. You can see the closet temperature cycle between the 2 boundaries I set for the fan. You can also see smaller peaks that are due to my failsafe of turning the fan on every hour regardless of temperature (I have code in misterhouse that checks if the temperature is low enough and turns the fan off, as well as turns it on when it reaches 85F, but it also gets turned on from cron on the hour as a backup if misterhouse or temperature sampling craps out).

It's also fun to see how the attic temperature changes as I exhaust heat from the computer closet into it, and how it even affects the HVAC duct a little bit, showing how those ducts, despite all their insulation, still leak temperature somewhat (the big peak in the duct is of course the HVAC system turning on).

measure the temperature in the main HVAC zone to see how hot the air I get from it is, or how cold air from our AC system is (and do the same for master bedroom subzone). For that, I have the hot air side (115F, which is lower than I thought) but not the cold air side yet since we haven't run the air conditioning since I've had the duct temperature monitoring.

Anyway, that's enough graphs for now, if you want more (and make your own), here is a link to a list of all the graphs.

1-Wire Setup

For temperature logging throughout the house and outside, the recommended solution is to use Maxim/Dallas 1-wire network (which is really 3 wires: ground, signal, and 5V which can be shunted back to ground if you don't need to measure extreme temperatures).

I'm not going to give a full description of everything since others have already done this better than me, but there is a fair amount of outdated information out there, so let me fix some of it (written as of 2009/12):

I went with a DS9097U maxim/dallas 1-wire serial adapter (I originally got the USB based DS9490R-A which is really a DS2490 chip, but I found it was less reliable with long networks, and it would just lock up and require a full power cycle with it had errors). If you decide to go with the USB version, since it is known to lockup with at least digitemp (maybe owfs is able to recover it), you will need to detect lockups and do a full USB power cycle to come back to life (i.e. you need to get a rare USB hub that is externally powered and allows power cycling. See my friend Andrew's blog if you need to get a USB hub that can power cycle ports).

DS9490R-A aka DS2490 USB 1wire adapter plugged in powered USB hub, and DS9097U serial adapter plugged into USB to serial adapter

If you think you should get the USB version because you don't have serial ports, a USB-serial adapter like in the picture above is only $20 or so. I really do prefer the reliability of the serial adapter and picked up a USB to 4 serial port adapter for $45 on ebay.

As far as software goes, you can use both digitemp or owfs. Owfs supports a few more devices and uses a cool filesystem interface, but this requires kernel support. Digitemp works well enough for most cases and is more portable to random devices, including my linksys router. I picked digitemp and will give examples for it below, but owfs is a fine choice too if it works for you.

Where to buy from?

First, you have the option of buying $20 modules that have the transistor looking temperature sensor inside (easier to wire) to just the temp sensor part for less than $5 and you have to wire it yourself. Your call.

A few places I found:

http://www.aag.com.mx/aagusa/index1.html, a good place to find the TAI8595 6 Channel 1-Wire HUB.
http://www.ibuttonlink.com/, for instance you can find the DS18B20 there.
Hobby Boards only seems to have the old DS18S20 but look for DS18B20 later or pick them up from ibuttonlink. They also have the DS9097U serial adaptor, and the 6 channel hub. Warning: Hobby Boards does not accept any returns, so know what you need before you buy it. They do have a nice 1-Wire Tutorial and list of equipment available.
Another source still: Newark.com

For temperature sensors, I started with the oft recommended DS18S20, but found out that the DS18B20 are actually much quicker to poll, so this is what people should buy today (they are fully compatible with digitemp and owfs). Oh, and the -PAR models are the same but with pins 1 and 3 hotwired so that you don't have to do it yourself (I've found it not necessary to use a more complicated powered network even for my long network, it's only required to sense extreme temperatures which you are unlikely to care about).

Note that all 4 kinds (DS18S20, DS18S20-PAR, DS18B20, and DS18B20-PAR) look exactly the same and work the same with digitemp and owfs. See this page for a comparison of the different sensors and you can read this page on PAR vs non -PAR.

for debugging, I used those cat-5 splitters, which do come in handy when I have to debug the network (unfortunately, the whole 1-wire network can go bad if something is wired backwards or there is a short, and you will lose part of the network if a miswire causes the circuit to be open). This is where they come in handy, the picture below shows 3 temp sensors wired in parallel to the splitter (I also label their serial number on a piece of tape for easier reading).

There is a big warning: all phone cords I've found are crossover cables as opposed to straight through, so if you use one, your pins will be reversed and you'll have to wire accordingly. See this picture of the USB 1wire adapter plugged into a DS18S20 where I had to reverse the pins for it to work due to the crossed phone cable:

the center pair is reversed due to the phone cable, the middle pin is actually signal, while 1 and 3 are ground and power shunted together

You can find wiring details on this page, or in a nutshell, my cheat sheet below:

pin 2 (RJ11)/pin 4 (RJ45) -> 1-W/DATA (DS18B20 pin  2 )	blue wire
pin 3 (RJ11)/pin 5 (RJ45) -> GND+VDD  (DS18B20 pin 1+3)	white wire

Cat 5 wiring to a plug where 1-wire is looped back from the blue pair to the green pair:
3 1-W back -> green/white
4 1-W -> blue
5 GND -> blue/white
6 Gnd back -> green

Note that the Maxim documentation gives you incorrect pin numbers because they count RJ11 pins in the opposite order from what's standard as per wikipedia.
I personally made my own cat-3 cable with my crimping tool and simple wire. That's the best way to know it's wired right (and you only need 2 wires anyway, the center pair)

I built a network that's about 1000ft long by using existing cat5 cables in our house, sending the signal down one pair, and bringing it back another pair (this doubles the distance, but works for me and saves my buying a 1-wire hub which would otherwise be required for this).

you can see my Cat-5 cables coming from multiple places in the house and being daisy chained together (I use the blue pair to get there and the green pair to come back

The other trick is to keep good track of the DS18B20s serial numbers as you install them since it will be hard to make out what is what otherwise when everything is installed later.

useful links

Digitemp Setup

Update at this point, I would not recommend using digitemp anymore unless you only ever plan on using DS18S20 and DS18B20 temp sensors, as this is mostly what digitemp supports (it does support one single humidity sensor too, but only with the serial interface, not the USB one). Long story short, if you're planning on doing anything more with 1-wire, you should invest the little extra time and set yourself up with owfs instead, which I've described in my later Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti blog post.

First: why would I use digitemp and log to a file from cron (which I then re-read and process for graphing and other uses) instead of using built in support in misterhouse? The reason turns out to be that I just don't want to lose temperature samples if misterhouse dies or if I'm hacking on it and have it down for a while. Simpler is better and more reliable in this case :)

USB:

On linux you may have to unload the kernel support for this USB adapter: rmmod ds2490

Then do a quick scan: gargamel:/tmp# digitemp_DS2490 -w DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Found DS2490 device #1 at 003/005 Turning off all DS2409 Couplers ... Devices on the Main LAN 1080BAED010800EA : DS1820/DS18S20/DS1920 Temperature Sensor 28D813060200000F : DS18B20 Temperature Sensor 812C282900000044 : Unknown Family Code

Then generate a config file: gargamel:/tmp# digitemp_DS2490 -i DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Found DS2490 device #1 at 003/005 Turning off all DS2409 Couplers ... Searching the 1-Wire LAN 1080BAED010800EA : DS1820/DS18S20/DS1920 Temperature Sensor 28D813060200000F : DS18B20 Temperature Sensor ROM #0 : 1080BAED010800EA ROM #1 : 28D813060200000F Wrote .digitemprc

Here you can edit/save the file, and running -a will use the config file in the current working directory: gargamel:/tmp# digitemp_DS2490 -a DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Found DS2490 device #1 at 003/005 Dec 26 22:26:05 Sensor 0 C: 24.31 F: 75.76 Dec 26 22:26:07 Sensor 1 C: 26.19 F: 79.14

The serial adapter works almost the same way, you just have to specify the serial port until it's written in the configuration file:

gargamel:/tmp# digitemp_DS9097U -s /dev/ttyS1 -w DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Turning off all DS2409 Couplers ............ Devices on the Main LAN 10A8D1ED0108009C : DS1820/DS18S20/DS1920 Temperature Sensor 10A8D1ED0108009C : DS1820/DS18S20/DS1920 Temperature Sensor 102223EF0108009B : DS1820/DS18S20/DS1920 Temperature Sensor 1052D1ED01080021 : DS1820/DS18S20/DS1920 Temperature Sensor 10D1D0ED0108005F : DS1820/DS18S20/DS1920 Temperature Sensor 10F9F3EE01080076 : DS1820/DS18S20/DS1920 Temperature Sensor 1045D2ED010800B1 : DS1820/DS18S20/DS1920 Temperature Sensor 10E5E7ED010800BB : DS1820/DS18S20/DS1920 Temperature Sensor 285CFC0502000004 : DS18B20 Temperature Sensor 28DD71C701000024 : DS18B20 Temperature Sensor 283359C7010000D8 : DS18B20 Temperature Sensor 2857B65902000062 : DS18B20 Temperature Sensor 09C7BE600500009E : DS2502/DS1982 1Kbit Add only memory

gargamel:/tmp# digitemp_DS9097U -s /dev/ttyS1 -i DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Turning off all DS2409 Couplers ........... Searching the 1-Wire LAN 10A8D1ED0108009C : DS1820/DS18S20/DS1920 Temperature Sensor 1094A2ED0108002F : DS1820/DS18S20/DS1920 Temperature Sensor 102223EF0108009B : DS1820/DS18S20/DS1920 Temperature Sensor 1052D1ED01080021 : DS1820/DS18S20/DS1920 Temperature Sensor 10D1D0ED0108005F : DS1820/DS18S20/DS1920 Temperature Sensor 10F9F3EE01080076 : DS1820/DS18S20/DS1920 Temperature Sensor 10E5E7ED010800BB : DS1820/DS18S20/DS1920 Temperature Sensor 285CFC0502000004 : DS18B20 Temperature Sensor 28DD71C701000024 : DS18B20 Temperature Sensor 283359C7010000D8 : DS18B20 Temperature Sensor 2857B65902000062 : DS18B20 Temperature Sensor ROM #0 : 10A8D1ED0108009C ROM #1 : 1094A2ED0108002F ROM #2 : 102223EF0108009B ROM #3 : 1052D1ED01080021 ROM #4 : 10D1D0ED0108005F ROM #5 : 10F9F3EE01080076 ROM #6 : 10E5E7ED010800BB ROM #7 : 285CFC0502000004 ROM #8 : 28DD71C701000024 ROM #9 : 283359C7010000D8 ROM #10 : 2857B65902000062 Wrote .digitemprc gargamel:/tmp# digitemp_DS9097U -s /dev/ttyS1 -a DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane GNU Public License v2.0 - http://www.digitemp.com Dec 26 22:30:39 Sensor 0 C: 16.88 F: 62.38 Dec 26 22:30:40 Sensor 1 C: 8.69 F: 47.64 Dec 26 22:30:41 Sensor 2 C: 13.44 F: 56.19 Dec 26 22:30:43 Sensor 3 C: 16.19 F: 61.14 Dec 26 22:30:44 Sensor 4 C: 10.69 F: 51.24 Dec 26 22:30:45 Sensor 5 C: 13.25 F: 55.85 Dec 26 22:30:46 Sensor 6 C: 9.06 F: 48.31 Dec 26 22:30:47 Sensor 7 C: 15.44 F: 59.79 Dec 26 22:30:48 Sensor 8 C: 5.25 F: 41.45 Dec 26 22:30:49 Sensor 9 C: -8.75 F: 16.25 Dec 26 22:30:50 Sensor 10 C: 17.25 F: 63.05

The trick is to save your .digitemprc in /etc/digitemprc, and then I call digitemp_DS9097U -a -c /etc/digitemprc | /var/local/scr/digitemp_rename.
Below are my configs:

gargamel:/tmp# cat /etc/digitemprc TTY /dev/DS9097U READ_TIME 1000 LOG_TYPE 1 #LOG_FORMAT "%b %d %H:%M:%S Sensor %s C: %.2C F: %.2F" LOG_FORMAT "%Y/%m/%d %H:%M:%S Sensor %s F: %.2F" CNT_FORMAT "%b %d %H:%M:%S Sensor %s #%n %C" HUM_FORMAT "%b %d %H:%M:%S Sensor %s C: %.2C F: %.2F H: %h%%" SENSORS 14 ROM 0 0x10 0x22 0x23 0xEF 0x01 0x08 0x00 0x9B ROM 1 0x10 0xA8 0xD1 0xED 0x01 0x08 0x00 0x9C ROM 2 0x10 0x94 0xA2 0xED 0x01 0x08 0x00 0x2F ROM 3 0x10 0x52 0xD1 0xED 0x01 0x08 0x00 0x21 ROM 4 0x10 0xD1 0xD0 0xED 0x01 0x08 0x00 0x5F ROM 5 0x10 0xF9 0xF3 0xEE 0x01 0x08 0x00 0x76 ROM 6 0x10 0x45 0xD2 0xED 0x01 0x08 0x00 0xB1 ROM 7 0x10 0xE5 0xE7 0xED 0x01 0x08 0x00 0xBB ROM 8 0x10 0x5D 0xE1 0xED 0x01 0x08 0x00 0x81 ROM 9 0x28 0x5C 0xFC 0x05 0x02 0x00 0x00 0x04 ROM 10 0x28 0x36 0xA3 0x59 0x02 0x00 0x00 0x38 ROM 11 0x28 0xDD 0x71 0xC7 0x01 0x00 0x00 0x24 ROM 12 0x28 0x33 0x59 0xC7 0x01 0x00 0x00 0xD8 ROM 13 0x28 0x57 0xB6 0x59 0x02 0x00 0x00 0x62

gargamel:/tmp# cat /var/local/scr/digitemp_rename #!/bin/bash

# the 2nd sed of seds does nothing, it's just there for me to keep track of mappings

sed -e "s/Sensor 10/13 MBR_ZoneVent/" -e "s/#10 : 2836A34902000038//" -e "s/Sensor 11/51 Garage_Fridge/" -e "s/#11 : 28DD71C701000024//" -e "s/Sensor 12/52 Garage_Freezer/" -e "s/#12 : 283359C7010000D8//" -e "s/Sensor 13/56 Hall_Closet/" -e "s/#13 : 2857B65902000062//" -e "s/Sensor 0/15 Garage/" -e "s/#0 : 102223EF0108009B//" -e "s/Sensor 1/11 Family_Room/" -e "s/#1 : 10A8D1ED0108009C//" -e "s/Sensor 2/22 Roof/" -e "s/#2 : 1094A2ED0108002F//" -e "s/Sensor 3/12 Living_Room/" -e "s/#3 : 1052D1ED01080021//" -e "s/Sensor 4/32 Outdoors_Crawlspace/" -e "s/#4 : 10D1D0ED0108005F//" -e "s/Sensor 5/31 Crawlspace/" -e "s/#5 : 10F9F3EE01080076//" -e "s/Sensor 6/55 Computer_Closet/" -e "s/#6 : 1045D2ED010800B1//" -e "s/Sensor 7/23 Outdoors_Roof/" -e "s/#7 : 10E5E7ED010800BB//" -e "s/Sensor 8/21 Attic/" -e "s/#8 : 105DE1ED01080081//" -e "s/Sensor 9/14 MBR_FloorVent/" -e "s/#9 : 285CFC0502000004//"

If you ever see this:

gandalf:~# digitemp_DS2490 -a
DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane
GNU Public License v2.0 - http://www.digitemp.com
Found DS2490 device #1 at 001/018
Oct 10 21:22:39 Sensor 0 C: 26.44 F: 79.59
CRC Failed. CRC is 63 instead of 0x00
CRC Failed. CRC is 63 instead of 0x00
CRC Failed. CRC is 63 instead of 0x00

It is likely because the digitemprc you are using is listing a 1wire device that is not visible on the network.

This is my somewhat complicated cronjob to gather digitemp temps and HVAC status and temp from misterhouse:

 * * * * * root alarm 25 digitemp -q -a -c /etc/digitemprc | grep Sensor | digitemp_rename | sort >> /var/log/temperatures

 * * * * * root DATE=`date "+\%Y/\%m/\%d\ \%H:\%M"`; sleep 45; (tail -100 /var/local/src/misterhouse/data/logs/thermostat.log | 
 grep 'Omnistat RC' | tail -2 | sed -e "s#^\(..\)/\(..\)/\(....\)#\3/\2/\1#" -e "s/, HVAC.*//" 
    -e "s/  Main Omnistat RC-90: Indoor temp is /01 Corridor F: /" 
    -e "s/  MBR Omnistat RC-80: Indoor temp is /02 MasterBR F: /" ; 
 tail -100 /var/local/src/misterhouse/data/logs/thermostat.log | grep 'Omnistat RC' | tail -2  | 
 sed -e "s#^\(..\)/\(..\)/\(....\)#\3/\2/\1#" 
    -e "s/  Main Omnistat RC-90:.*HVAC Command: /05 Corridor_HVAC: /"  
    -e "s/  MBR Omnistat RC-80:.*HVAC Command: /06 MBR_HVAC: /" -e "s/,.*//" 
    -e "s#fan/##" -e "s#off/##"  -e "s/off/0/" -e "s/fan/0/" -e "s/cool/-1/" -e "s/heat/1/" )| 
 grep "$DATE" | sort >> /var/log/temperatures

End result looks like this:

2009/12/27 08:44:03 15 Garage F: 54.27
2009/12/27 08:44:04 11 Family_Room F: 62.26
2009/12/27 08:44:06 22 Roof F: 55.96
2009/12/27 08:44:07 12 Living_Room F: 61.02
2009/12/27 08:44:08 32 Outdoors_Crawlspace F: 49.89
2009/12/27 08:44:09 31 Crawlspace F: 56.08
2009/12/27 08:44:10 55 Computer_Closet F: 75.76
2009/12/27 08:44:11 23 Outdoors_Roof F: 48.09
2009/12/27 08:44:12 21 Attic F: 57.42
2009/12/27 08:44:13 14 MBR_FloorVent F: 69.69
2009/12/27 08:44:14 13 MBR_ZoneVent F: 70.47
2009/12/27 08:44:15 51 Garage_Fridge F: 40.77
2009/12/27 08:44:17 52 Garage_Freezer F: 16.81
2009/12/27 08:44:18 56 Hall_Closet F: 62.71
2009/12/27 08:44:00 01 Corridor F: 65
2009/12/27 08:44:00 05 Corridor_HVAC: 0
2009/12/27 08:44:30 02 MasterBR F: 64
2009/12/27 08:44:30 06 MBR_HVAC: 0

Here is a link to my cacti_owfs script I use to read the above log and for graphing in cacti or rrdtool, or for getting a data sample that I can use in misterhouse (more generally, you want to see my Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti page as well as my Gatewaying 1-wire, XPL (Oregon Scientify Weather), Brultech ECM1240 Power Data, and Brand OneMeter Data to cacti page for cacti integration page.

1-Wire Applications

I keep track of my fridge vs freezer temperatures, which is hard to otherwise get on most fridges.

For now I only use the temperature to turn the fan in the computer closet on and off (with a backup that turns it back on every hour just in case misterhouse died), but I am hoping to get more automation of the sort like getting outside air in at the right times of the day to regulate temperature for virtually free when possible (as well as recirculate the air).
You can see the graph above for my computer closet temperature as controlled by an exhaust fan.
Update: I did get around to installing an outside air cooling/heating system, the details are on my Booster Fans and Heating/Cooling with Outside Air with Misterhouse page.

Ideally, instead of using a zone board to turn on the HVAC when the subzone thermostat wants to cool/heat the master bedroom, I would have used misterhouse to tell the secondary thermostat to trigger the HVAC through the first one, but my code wasn't ready then and more importantly I felt that maybe basic functionality in the house should not rely on a computer that I'm the only one who can fix :) (it would also have sucked for the next people if we sell the house and had moved out since the HVAC system would have not worked properly without my computer). See more images for Temperature monitoring and graphing with 1wire devices, digitemp, misterhouse, and cacti

Brand One Powermeter, Solar and Power Monitoring with Cacti and Real Time PG&E Time of Use price calculation

2009-11-08T16:00:00Z

Update: I have a much better solution (and meter), the ECM1240 from Brultech, read about it here.

Ok, the title is a mouthful, but that's why it's been about 6 months since I started and I have results to show for it only now.

My goals were to get:

a daily summary of power use per hour

have enough data to recreate a PG&E bill on a daily basis before we get it (to be able to predict whether we're using electricity in the right amount and at the right time before getting the bill).

more importantly, be able to drill a bill down to the day and the hour (we have data updates every 2mn) and see how much more power the house is using if the dishwasher is on, or the electric toaster oven.

get a real sense of how much our AC is running and how much it costs.

For this, while ordering the Solar Panels (aka PV system) from Cobalt Power, I asked for a monitoring system and the only one they found at the time that would allow monitoring of both the PV production and the PG&E Meter was a third party One Meter from One Brand Electronics.

It looks like this:

the meter that does gathering and resending to my monitoring server

the probe gathering boxes (gets volts and amps)

this shows the CTs that measure current from their induced magnetic field

The voltage probes are simply connected to the lines and those are precise. Unfortunately, I don't care about voltage nearly as much as current sinc ethe voltage tends to be known and the current is what affects your bill.

It's a reasonably easy solution to add to any setup during or after the fact, but the low points with the Brand Electronics One Meter solution are:

by design the current measured is inexact. I see current used on the PV inverter when it's not supposed to use much of anything, and of course I also see random small currents in either direction on the AC when it's off.

while the data gathered is good enough for general trends, longer term use has shown that it can be off by as much as 40Kwh/month on my PG&E Meter, which is a fair amount.

the One Meter interface is really bad. I'm a linux CLI guy, but it is bad. It is impossible to change its basic configuration like what it sends over telnet and at what interval without having its firmware reburnt by onebrand, but get this: you can't even change the IP address it'll talk to either without sending it back.

it has a serial interface for configuration that is quite poor and it'll dump old data you might have missed in a different format that what it sends over telnet to make your life harder.

it is a very slow microcontroller that is so slow that it takes over one minute to send 12 samples of data

it is of course unable to set its own time over ntp and is unaware of DST, so I just put mine on UTC and I fix things on the server side.

last, but not least, it will randomly corrupt itself and send totally bogus data points. Thankfully it however recovers at the next sample. I had to write fairly complex code to analyse the data samples, detect and throw away bad ones.

Ok, so are you sold yet? :)
Well, it's not that bad now since I went through the effort of writing the code to deal with all this. So, if you were to get one, you'd be up and running pretty quickly if you can hack perl, the language I used for my magic script that does all the work (see the bottom of the page for source code).

The above script took a fair amount of time to write since outside of working around quirks in the One Meter output, I wrote in the PG&E billing logic for California and it is able to output per hour per day production and use, as well as equivalency in dollars. Converting into dollars make sense since with TOU (time of use pricing) you can end up with days where your used more than you produced, but the end bill is still negative.

This is a typical example of a summer day (July 8th picked at random):

Yellow shows partial peak rates, where it's good to have the meter run backwards, and red for peak rates when it's even better. Unfortunately the solar panels are facing south east and south instead of south west where they would produce more when electricity is worth more.

That is a more interesting example in September showing that despite having used more electricity in a day than what was produced, the daily bill was still negative thanks to the time of use magic, even it our case where we only get partial benefit from TOU due to the sub optimal south east angle for some of our panels:

If you want to see other days at random, you can pick them from here.

Now, a per hour text output is useful to see how we did on a given day, but it does not let you see your power usage in the last 10mn after you turned something big on or off, or how all of last week looked, or see a month's trend at a glance.

This is where cacti comes in. I spent a fair amount of time looking for a graphing solution that would keep all my data and let me zoom on any portions I wish. Quite frankly, the fancy widget that google uses in google finance would have been what I was really hoping for, but baring that, cacti came like a reasonable alternative.

So, I came up with a compound graph that looked reasonable, and the option to see all the graph items separately.

So since this is about solar panels, one of my questions to Cobalt Power was why my system was spec'ed for 5Kwh at peak and why I was typically seeing 4.5-4.7Kwh at best. The reason is that 2/3rd of the panels are pointing south-south-east and the remaining 3rd is pointing due south.
What this means is that our system doesn't actually peak at one time and then drop off. The graph clear shows that the PV production takes longer to peak and then stays at that lower peak a bit longer before dropping off more sharply. So, the production at the end of the day is still the same, but just with a slightly longer and slightly lower peak. Too bad the peak is on the morning side as opposed to the afternoon side when the electricity is worth more.
You'll also notice the sharp start which is when the sun goes high enough to reach the roof over the neighbours' houses across the street. You'll also notice the jigsaw drop on the way down as the sun starts to get hidden behind some branches in the high trees blocking our west view.

This graph and the graphs below are selected in zoom mode, so you can use the mouse cursor to draw a rectangle on a time region and the graph will refresh on the time slice you selected. Note that you can also zoom out by clicking the graph with the right button:

Below is a closeup of all the colors showing AC working hard in bursts while the black PG&E meter line goes from sometimes negative to always positive the green area of solar panel energy offset goes down to 0. Blue then shows the basic house energy usage with the red peaks on top showing AC tripling the house electricity use when it's running.

In other words: the red line is the AC use, the blue is the rest of the house use, red line goes on top of house use gives the red area which is total house use. You then remove green which is production from the solar panels and you end up with the black line which is what the PG&E meter actually sees. Confused yet? :)

Here's a typical day without AC use since we weren't home:

Here's a very warm day with AC (keep in mind that you can zoom in by selecting a time slice with the mouse):

and a glance at just AC use during a few hot days in July:

Here is a link to an historical view of this graph at multiple time sample levels. Try the zoom function (magnifying mirror next to the graph) on the yearly graph at the bottom and you'll be able to zoom on random time in the past.

Another view is each probe graphed separately for a less cluttered view. Note that the AC view is pretty useless right now as it's only showing noise picked up by the coils, but if you click on it, you'll get the multiple timerange view as above and can view more interesting months, like the July zoom above.

What else?
Well, I had written some somewhat complex code to actually keep up of production for each tier and find out if I went into tier 2 to 5 (which pay or cost more per unit of energy), but because the One Meter gives me somewhat inaccurate readings, especially on the PG&E side, the numbers were just too far off to compute a bill that was close enough to the actual bill I was getting. So, I unfortunately had to give that part up.
It would otherwise have been nice to know in advance if my use or production for a given tier was going to exceed tier 1 and possibly adjust electricity use accordingly, but I'll need a more accurate measuring device than the One Brand one.

All this ended up being a fair amount of work, which with not that much extra work could have been a service that all solar companies sell. I think a few do, but I'm not sure if it's as complete as what I did, or if it is, please let me know so that I can compare with their work.

Anyway, if it's useful to you, here's the parsebrandpower script. Please note that if you were planning on taking it and selling service based on my work, it is protected by the GPL 3 copyright which you must understand and apply if you are going to use it. I also request that you contact me and let me know if you're going to use the code.
For more details on cacti integration, see my Gatewaying 1-wire, XPL (Oregon Scientify Weather), Brultech ECM1240 Power Data, and Brand OneMeter Data to cacti page. See more images for Brand One Powermeter, Solar and Power Monitoring with Cacti and Real Time PG&E Time of Use price calculation

Work on HAI Omnistat code for misterhouse

2009-08-04T15:00:00Z

We recently made a subzone for our bedroom which allows to send a crapload of air from our HVAC system to the bedroom at night so that it can get cooled or heated quickly without bothering to bring the rest of the house to temperature.
But to control and monitor all this, there was some half finished code in misterhouse that I cleaned up, fixed up and improved. I now get reports of what's going on, can remotely control my HVAC system to change its setpoints, remotely and easily program its 48 time and setpoint settings via a web browser instead of the cumbersome panel interface, and of course I can log everything, like so:

07/08/2009 18:57:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 50, cool to 86, mode: program_auto 07/08/2009 18:57:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 18:58:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 50, cool to 86, mode: program_auto 07/08/2009 18:58:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 18:59:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 50, cool to 86, mode: program_auto 07/08/2009 18:59:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 19:00:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: fan/cool, heat to 62, cool to 74, mode: program_auto 07/08/2009 19:00:01 Main Omnistat State set to: cool_sp_change 07/08/2009 19:00:01 Main Omnistat State set to: heat_sp_change 07/08/2009 19:00:01 Main Omnistat State set to: current_output_change 07/08/2009 19:00:14 set cool to 76 07/08/2009 19:00:16 Main Omnistat State set to: cool_sp_change 07/08/2009 19:00:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 19:01:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 62, cool to 76, mode: program_auto 07/08/2009 19:01:00 Main Omnistat State set to: current_output_change 07/08/2009 19:01:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 19:02:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 62, cool to 76, mode: program_auto 07/08/2009 19:02:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 19:02:30 MBR Omnistat State set to: temp_change 07/08/2009 19:03:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 62, cool to 76, mode: program_auto 07/08/2009 19:03:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto 07/08/2009 19:04:00 Main Omnistat RC-90: Indoor temp is 74, HVAC Command: off, heat to 62, cool to 76, mode: program_auto 07/08/2009 19:04:30 MBR Omnistat RC-80: Indoor temp is 73, HVAC Command: off, heat to 50, cool to 88, mode: program_auto

You can see more on the Misterhouse HAI Omnistat Setup Page

Mythtv + Denon-AVR3808 Hack

2009-05-27T15:00:00Z

I had a minor issue where I have 4 hauppage remote controls and one on the kitchen counter that I sometimes use to control my mythtv when I'm in the kitchen.
The only problem was that sound is actually controlled by the Denon receiver and that the hauppage remote doesn't talk to the denon.

I finally found a fix for this, which somewhat justifies the extra money I spent for an ethernet connected receiver: I wrote an lirc configuration that effectively telnets to the denon to tell it to change volume when the hauppage receives a relevant remote code for that.

lircrc:

# Denon
# Mute
begin
prog = irexec
button = MUTE
repeat = 1
config = /var/local/scr/denonmute
end

begin
prog = irexec
button = Vol+
repeat = 1
config = /var/local/scr/denoncmd MVUP 3
end

begin
prog = irexec
button = Vol-
repeat = 1
config = /var/local/scr/denoncmd MVDOWN 3
end

denonmute:

#!/bin/bash

FILE=/var/run/denonmute

test -f $FILE || touch $FILE

# prevent bounces
[ $(( $(date "+%s") - $(stat -c "%Z" $FILE) )) -lt 2 ] && exit

if grep -q MUON $FILE 2>/dev/null; then
        CMD=MUOFF
else
        CMD=MUON
fi

echo -ne "$CMD\r" | nc -q0 denon 23
echo $CMD > $FILE

denoncmd:

#!/bin/bash

# $0 cmd [repeat]

for i in `seq 1 ${2-1}`
do
        echo -ne "$1\r" | nc -q0 denon 23
        sleep 0.2
done

exit

And there you go, I can now send volume requests to my mythtv and they get forwarded to my denon over TCP/IP. Pretty cool :)

Insteon Setup and my own Blender Defender

2009-03-16T15:00:00Z

I have been spending a fair amount of time learning how to setup and work with Insteon Home Automation recently, as a way to replace X10 I had in my old house. Insteon is meant to be close to 100% reliable when setup correctly.

I could write a lot about Insteon, but I already have spent around 10H doing so in the Misterhouse Insteon docs for linux HA I wrote. In the process, I also researched UPB and Z-Wave to decide which one was the best option and here is my Summary of X10 vs UPB vs Z-Wave vs Insteon.

my own Blender Defender

what an insteon dimmer switch looks like

my main power strip going to a filter, X10 and Insteon modems

X10 CM11a on the left, Insteon Filter on lower right and Insteon Modem (PLM) on upper right

The Insteon Filter allows me to plug my UPS and devices in the back of it without them messing with Insteon signals going to and coming from the powerline.

While most of the time was spent setting up misterhouse to control our home lights and create virtual scenes as well as manage our outside lights depending on motion sensor input, I also build our own Blender Defender to teach chat to stay off the counter where he likes to go explore in search of food :) (Blender Defender is not my idea, I just improved on the concept since mine had to work at night too, see the link for the original author who inspired me).

My blender defender was a bit harder to build because it had to work at night, and I had to include a motion sensor to turn the light on first before the camera could work and detect motion (which makes things a bit harder because the motion detection library has to deal with a change in light, which can look like motion).
So far, it seems to have worked, although it was a bit less satisfying, maybe also because our blender is missing the strobe lights and isn't as scary and chat seems to have learned the first time.

Anyway, here's the result, taken in the middle of the night with the living room light turned via Insteon thanks to the script (yes, the camera doesn't show the blender, but the cat has been coming by jumping from the couch next to the kitchen counter, and this is the only camera angle that stops the cat as soon as he crosses the threshold without triggering when he's at the same height on the couch side, which is allowed).

Just like the original blender defender, I also used the super cheap D-Link DSC-900 and wrote my own scripts to activate my insteon device and light (including the fun part of preventing the outside patio light from turning on on outside motion sensor events as the light through the window could have changed the kitchen picture enough to trigger the blender in the middle of the night).
The hard part in the setup was the motion sensor library tweaks. You'll want motion and I heartily recommend starting with my motion.conf file and look for the CHANGED tags as a guide to setup your own (please understand that eventually you'll need to tweak for your own room and camera, good luck with that :) ). See more images for Insteon Setup and my own Blender Defender

TV and remotes with linux

2004-03-25T16:13:00Z

Current Music: DJ Tiësto - Live At Innercity - Norefjell
Current Mood: Yeah another long time project finally completed

So, those who've visited my house know that I have a sick system where I use
X10 wireless remotes, that get picked up by a receiver in the wall, which
transforms the signals from RF to electrical signals, which then get picked up
by a receiver in my linen^H^H^H^H^Hcomputer closet, which transforms this into a command for my serial port, which is read by a linux daemon, which in turn runs commands of my choosing, like changing the music or the volume (it's especially awesome under the shower, because I can use a throwaway wireless remote, and still control the music. I'm sure you'll agree that it is very important to be able to do so ;-)

Anyway, I also have another PC that is my video server, and plays files from another server (my disk server), and sends the video in a S-Video cable that goes under the house and reaches my TV.
So far, so good, except that I had to use my laptop on my couch to control the video and pause it or whatever (this turned out not to be a great problem since my laptop is always on my couch, but I digress).

So, I have some TV capture card with an IR receiver, and an IR remote. With a little work, I found and setup some linux software to receive the IR codes from the remote, and pass them on as commands to mplayer, a video player on linux.
This wasn't bad, except that I have to point the IR remote in my back, towards the PC, which is nowhere near the TV, and it wouldn't even work at all if I can see the TV but I'm a place in the room where my IR remote can't reach the IR receiver.
So, I had that working a couple of years ago, but I never really used it because it was a clearly inferior and imperfect solution that no self respecting geek would bring himself down to using :)

That's where my mouseremote comes in. This handy little sucker looks like a 5 way learning remote, also supports sending X10 events (i.e. to control my lights, and also my music from the closet), and then it also sends two more sets of RF signals that get picked up by another computer. The first set allows me to move the mouse on my TV screen for the PC I use to play videos, and the second set allows me to send random keys from the remote and have them used by the PC.
This is where the interesting part comes in: using those keys to do interesting stuff, like controlling the video so that I can just use my mouseremote like a regular remote, except that I'm talking to a PC that's not even within line of sight.
I wouldn't want to further bore you with technical details, and you can find them here should you really be curious (considering that it took me a little while to get it working, I documented it so that the next guy who tries this doesn't have to spend as much time than me). I'll just leave you with the (admittedly somewhat insane) list of things that happens in real life when you press play:

the mouseremote sends an RF signal with the key I pressed
multimoused on linux reads from the serial port and receives a translation of the RF signal
it generates mouse events into the /dev/mumse fifo
it separates and sends keypresses (like DVD+1) to /dev/x10fifo
MouseRemote reads from the fifo and executes commands as directed in MouseRemote.conf
MouseRemote.conf calls irsend, like so ' irsend SEND_RELAY "0000000000001005 00 5" Hauppauge', which tricks the patches lircd to think I used my IR remote to send this code
I'm running a patched lircd that understand this special trick, and relays it to its fifo as if it were coming from the infrared remote
mplayer, running in lirc mode, reads from the lircd fifo, parses ~/.lircrc, receives the fake IR code, and pauses or resumes the video

See, it's really simple :)
I only got that working last night once I got the special evil trick to generate fake IR events in lircd, as if I were using my IR remote. We'll put aside the fact that I've owned the remote for mmmh, 4 years now :-)
Eh, at least I get points for documenting all this , so as to make it easier for the next guy...

Anyway, that's when you get to witness someone next to me in my couch, me pressing play or pause on my remote, the video stopping or starting as appropriate, me displaying a huge grin on my face, and the person next to me wondering "what's up with him, he pressed pause, the video paused, what's so special about that?".
If they only knew ;-)))

This is where I also get to say a few words about linux vs windows.
With windows, I would definitely have spent less time on all this: either I would have found some complex software that someone wrote and that happened to do what I needed, or I would have been entirely out of luck.
With linux, it took me a fair number of hours to make this all work (10 or so), but once I found that the main pieces existed (IR and RF readers) I knew that I could write the missing glue to make it all work together. In its own way, if you are into this, it's really rewarding to not only know it all works, but to have been able to make it work and piece it up together...

Eh, everyone needs a hobby, or two, or three, or... Oh, whatever, just let it be :)

Actually, if I get really bored, I'll have to setup freevo and MythTV on one, or several of my PCs, to turn them into more tivos (although that would probably also mean that I'd have to shell out some bucks to buy a small and quiet computer, and I have that stupid bet with myself that I have so much hardware already that I don't need to buy more computers (and I haven't bought a computer in more than 5 years, everything I have is from excess spare parts that I put together).
Anyway, I already have a Tivo and ReplayTV, and I don't *really* need to be watching more TV, and I don't really have the time to get bored, but you never know :)