Gatewaying 1-wire, XPL (Oregon Scientify Weather), Brultech ECM1240 Power Data, and Brand OneMeter Data to cacti

Cacti related scripts

Usage info can be found on my linuxha blog for now, specifially:

My older page on Temperature monitoring and graphing with 1wire devices, digitemp, misterhouse, and cacti
My newer page on Temperature, moisture, humidity, and UV monitoring and graphing with 1wire devices, owfs, and cacti
Detailled solar stats in my Brand One Powermeter, Solar and Power Monitoring with Cacti and Real Time PG&E Time of Use price calculation.
Daily Power and Solar Stats (at the year level).
Fine grained house-wide power monitoring with Brultech ECM1240, ecmread.py (with net metering support), and graphing with cacti (at the year level).

And here are the magic scripts:

Owfs

read_owfs reads owfs data from a symlink tree in /etc/owfs/ and populates a digitemp looking file in /var/log/temperatures.
cacti_owfs reads a digitemp or owfs derived /var/log/temperatures file and converts it into rrdtool data or cacti compatible query data.

xPL
parse_rfx-xpl gets data from rfx-xpl (read from a rfxcom device), and parses the weather data into a format suitable for logging and further processing.

cacti_xpl an xpl.log file generated from parse_rfx-xpl and converts it into rrdtool data or cacti compatible query data.

Example output from parse_rfx_xpl will look like this:

2010/08/21 15:35:53: STR918,WGR918 addr: 232, chan: 0 wind: 80° E, speed 2.60 m/s / 5 kts average 2.20 m/s / 4 kts battery level 100% 
2010/08/21 15:36:03: THGR918 addr: 225, chan: 1 temp: 23.30°C / 73.94°F hum: 38 % dry battery level 100% 
2010/08/21 15:36:07: STR918,WGR918 addr: 232, chan: 0 wind: 52° NE, speed 2.40 m/s / 4 kts average 2.20 m/s / 4 kts battery level 100% 
2010/08/21 15:36:11: RGR126,RGR682,RGR918 addr: 147, chan: 0 rain: total 1163 mm, 0 mm/hr, count 1 
2010/08/21 15:36:17: BTHR918N,BTHR968 addr: 230, chan: 0 temp: 23.50°C / 74.30°F hum: 54 % comfort baro: 1015 hPa / 29.97 inHg, forecast: sunny 
2010/08/21 15:36:21: STR918,WGR918 addr: 232, chan: 0 wind: 20° NNE, speed 2.40 m/s / 4 kts average 2.20 m/s / 4 kts battery level 100% 
2010/08/21 15:36:35: STR918,WGR918 addr: 232, chan: 0 wind: 1° N, speed 2.40 m/s / 4 kts average 2.40 m/s / 4 kts battery level 100%

To learn more about xPL and rfx-xpl, see my misterhouse page on the topic.

Brultech ECM1240

ecm.init starts ecm monitoring.
ecmread.py is the script that talks to an ECM1240 and dumps its data to a file.
ecm_addnames1 takes ecmread.py output and modifies it to match your local setup.
parseecmpower reads the dumpfile from ecmread.py and can generate an rrdtool file, as well as generate daily and monthly power reports.

Brand One Power Meter
readtelnet receives data from a Brand OneMeter (to be used as a login shell), adds it to a file and feeds it into a cacti/rrdtool file
parsebrandpower reads the dumpfile above and can generate an rrdtool file, as well as generate daily and monthly power reports.

A few examples:
Monitoring my house temperatures and effect of HVAC:

Monitoring my fridges and freezers:

Brand One Power Meter House, AC, and Solar Panel data:

Quick cacti config howto

I can't give a full cacti tutorial here, you should first go to http://docs.cacti.net/manual:087.

But here is a post I sent to the cacti-user mailing list which gives an outline of what I did. If you have cacti questions, though, please post to the proper forums instead of Emailing me directly, thanks :) First thing to know is that cacti is really a gui around rrdtool, so read up on rrdtool a bit, try the tutorial just to get a feel for how rrdtool works: http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html

Then go through the cacti docs, understanding that portions were meant for snmp and complex templates which you won't apply much in this case: http://docs.cacti.net/manual:087

Note that you will definitely understand cacti better if you understand how to do basic rrdtool operation without cacti (hence the first tutorial). What I have is in:

data input method / House temps:
script/command
/var/local/scr/cacti_owfs --cacti

which returns

Corridor:60 MasterBR:61 Corridor_HVAC:0 MBR_HVAC:0 Family_Room:60.35 Living_Room:58.55 MBR_ZoneVent:58.10 MBR_FloorVent:58.66 Garage:58.89 Attic:58.89 Roof:58.89 Outdoors_Roof:55.96 Crawlspace:56.41 Outdoors_Crawlspace:55.06 BR2:U BR3:U BR4:U Garage_Fridge:42.80 Garage_Freezer:18.16 Computer_Closet:81.84 Hall_Closet:59.56

you then configure the output fields to match the names.

At this point, you need to configure a data source:
House Temperatures Source:
(you'll want 'Turn On Data Source Debug Mode' in upper right corner)
Mine gives:

/usr/bin/rrdtool create \
/var/lib/cacti/rra/housetemp_17.rrd \
--step 60  \
DS:Attic:GAUGE:600:0:150 \
DS:MBR_FloorVent:GAUGE:600:0:150 \
DS:MBR_HVAC:GAUGE:600:-1:1 \
DS:Corridor_HVAC:GAUGE:600:-1:1 \
DS:Garage_Freezer:GAUGE:600:-50:150 \
DS:Garage_Fridge:GAUGE:600:0:150 \
DS:BR4:GAUGE:600:0:150 \
DS:BR3:GAUGE:600:0:150 \
DS:BR2:GAUGE:600:0:150 \
DS:Roof:GAUGE:600:0:150 \
DS:Outdoors_Roof:GAUGE:600:0:150 \
DS:Outdoors_Crawlspace:GAUGE:600:0:150 \
DS:Hall_Closet:GAUGE:600:0:150 \
DS:Computer_Closet:GAUGE:600:0:150 \
DS:Corridor:GAUGE:600:0:150 \
DS:Crawlspace:GAUGE:600:0:150 \
DS:Family_Room:GAUGE:600:0:150 \
DS:Garage:GAUGE:600:0:150 \
DS:Living_Room:GAUGE:600:0:150 \
DS:MasterBR:GAUGE:600:0:150 \
DS:MBR_ZoneVent:GAUGE:600:0:150 \
RRA:AVERAGE:0.5:1:2628000 \
RRA:AVERAGE:0.5:1440:10950 \
RRA:AVERAGE:0.5:5:3153600 \
RRA:AVERAGE:0.5:30:525600 \
RRA:AVERAGE:0.5:120:131400 \
RRA:MAX:0.5:120:131400 \
RRA:MAX:0.5:1440:10950 \
RRA:MAX:0.5:30:525600 \

Once you set that up, cacti will use rrdtool to create /var/lib/cacti/rra/housetemp_17.rrd

If you change things, you'll have to delete and recreate that file. You can also fix data by turning off 'Data Source Active', recreating the same rrd file as cacti would have (i.e. use the command above from the command line, make sure you use --start, and for the permissions/owner of the file after it's created). Then pre-fill it with your data dump which you got from cacti_owfs --cacti-dump or similar (useful when you add fields or you want to manually regenerate the file).
When you're done, re-enable 'Data Source Active'. This tells cacti to run its poller, which will call the cacti_temp command above, it takes the data fields and feeds them into an rrdtool update command for the rrd that was created with the command shown above (if things don't work, look at /var/log/cacti/*log and make sure the permissions on the rrd file are good, www-data for me).

At this point, your data is gathering but you don't see anything yet, see below: Note that I have non detault round robin numbers, my table in data sources/RRAs looks like this:

Name 					Steps 	Rows 	Timespan
Daily	(5 Minute Avg for 30Y) 		5 	3153600	86400 	
Daily	(5 Minute Avg for 5Y) 		5	525600 	86400 	
Hourly	(1 Minute Avg for 1Y) 		1 	525600 	14400 	
Hourly	(1 Minute Avg for 5Y) 		1 	2628000	14400 	
Monthly (2 Hour Avg/Max for 30Y) 	120 	131400 	2678400 
Weekly	(30 Minute Avg/Max for 30Y) 	30 	525600 	604800 	
Yearly	(1 Day Avg/Max for 30Y) 	1440 	10950 	31536000

cacti/rrdtool are really meant to lose resolution over time and cacti will want to have hourly daily weekly and monthly data to display all 4 graphs but if your hourly graph has all the data forever, rrdtool will use that when you zoom on hourly data 1 year in the past. See: http://docs.cacti.net/manual:088:8_rrdtool.03_big_rra#no_data_loss Go to graph management, and create a simple graph like this (this was by clicking on the "Turn On Graph Debug Mode" tool in upper right)

It's a bit tedious because you need to manually add all the fields through the GUI, which really takes a while (I do it while watching TV). The result for a triple graph is:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Garage vs Fridge & Freezer" \
--rigid \
--base=1000 \
--height=320 \
--width=800 \
--alt-autoscale \
--vertical-label="temperature (F)" \
--slope-mode \
--font TITLE:12: \
--font AXIS:8: \
--font LEGEND:10: \
--font UNIT:8: \
DEF:a="/var/lib/cacti/rra/housetemp_17.rrd":Garage:AVERAGE \
DEF:b="/var/lib/cacti/rra/housetemp_17.rrd":Garage_Fridge:AVERAGE \
DEF:c="/var/lib/cacti/rra/housetemp_17.rrd":Garage_Freezer:AVERAGE \
LINE2:a#009485FF:"Garage"  \
GPRINT:a:LAST:"                      Current\:%8.2lf %s"  \
GPRINT:a:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:a:MIN:"Minimum\:%8.2lf %s"  \
GPRINT:a:MAX:"Maximum\:%8.2lf %s\n"  \
LINE2:b#4444FFFF:"Garage Fridge"  \
GPRINT:b:LAST:"         Current\:%8.2lf %s"  \
GPRINT:b:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:b:MIN:"Minimum\:%8.2lf %s"  \
GPRINT:b:MAX:"Maximum\:%8.2lf %s\n"  \
LINE2:c#FFF200FF:"Garage Freezer"  \
GPRINT:c:LAST:"       Current\:%8.2lf %s"  \
GPRINT:c:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:c:MIN:"Minimum\:%8.2lf %s"  \
GPRINT:c:MAX:"Maximum\:%8.2lf %s\n"

For graphs like this one, I used a template so that I can make the graphs settings once and quickly apply them to all the data probes.

Last, but not least, you need to add graphs in 'graph trees' for them to show up.

It's a very quick outline, but I hope it helps :)

Email
Link to Home Page