Tuesday, October 31, 2006

VBScript to export Outlook contacts to import into HAL

This script exports your Outlook contacts into a format usable by the build_addrbook_dbf.pl script from yesterday. useage:

export_contacts_hal.vbs username

You'll need to edit the file to change this line to your Outlook profile user name:
if user="FirstUser" then

It is written to work with multiple user profiles. I have not tried it with only one user profile. Some instructions inside the script.

Monday, October 30, 2006

Perl script to build HAL addrbook.dbf

Building on the previous post, I've modified that script to create the HAL address book database from a text file. This can be used to import data from Outlook or whatever PIM you use. You just need to export your data into a CSV and massage it into the right format. Instructions in the file. Of course, backup your files before you play.

the script

Sunday, October 29, 2006

Perl Script to Build HAL Tasks Database

Here's a script I just finished to build the HAL tasks database. You can now edit your tasks in text and use this script to create the .DBF and .FPT files. No more kludgy HAL point and click editor :) It requires the XBase package from my previous post.

the script

Make sure you backup your HAL data before using this script!!!

Saturday, October 28, 2006

Power Consumption

One of the problems with all of the gadgets running in our house is it uses a lot of electricity to keep everything going. Starting in the summer of 2005, I began to work on reducing our electricity usage. The obvious things were changing out most of our light bulbs for compact fluorescent (and upgrading from standard X10 light switches to Leviton X10 switches for non-incandescent loads). Tons of Hawkeye motion sensors were sprinkled thoughout the house (they talk to the W800 RF receiver which starCOM interfaces to). Lights are turned off in all rooms now based on motion. This has worked well with 2 little kids running around the house leaving things on.

DC power supplies can use anywhere from 3-5 (or more watts) all day long. We probably have 50 or so of these in the house. Most of our chargers are on appliance modules that are on timer so they don't suck power all day. During the night when we all go to sleep, the system turns off the power to some TVs and modulators (which are on appliance modules). It also turns off a lot of those DC power supplies that aren't charging anything. It turns them back on in the morning.

The system knows the state of security system so when we leave the house, it forces all non-essential PCs to hibernate.

DS10 alarm sensors are put on some windows and talk to the W800. This way, the system knows that windows are open and will not turn on the HVAC. LM34 temperature sensors are distributed throughout the house, and the HVAC system is only activated when the coldest/warmest room needs it. Outside temperature sensors tell the system that it's no longer necessary to run HVAC and will remind us to open the windows, etc. In the summer, box fans are put in the windows when it's cool enough to open them. They're on appliance modules and turn off when the room they're in is cool enough.

I've also done a lot of server consolidation, going from 4 full time servers to only 2 right now. There's also been a fair amount of weather proofing done eventhough our house was brand new when we moved in.

Here's a graph of our electricity usage & cost since we moved in. You can click it for a larger view. We've cut it down about 50% from our highest peak usage.



For reference, we have all electric appliances, except a gas range and gas water heater.

Thursday, October 26, 2006

Reading HAL Databases

HAL databases are stored in a Fox Pro format. Why would anyone want to read those databases? Maybe to script the creation of tasks, macros & events? This post will get you halfway there - reading the databases. First off, you'll need Perl. Get ActivePerl if you don't have Perl already. After installing it, open a command prompt and start the package manager by running ppm. At the ppm prompt, type search DBD-XBase, then type install. After it finishes, type exit and now you're ready to write some code.

This is a very simple script that takes 1 argument (the .DBF filename) and dumps out the contents of the file, field by field.

# dumpdb.pl
use XBase;
$\ = "\n";
$, = ' ';
my $table = new XBase $ARGV[0] or die XBase->errstr;
@fields= $table->field_names;
for (0 .. $table->last_record) {
foreach $field (@fields) {
my ($deleted,$val) = $table->get_record($_, $field);
print "$field = $val" unless $deleted;
}
print "";
}
$table->close;

usage: perl dumpdb.pl FILENAME.DBF

Wednesday, October 25, 2006

Keeping xAP Jabber connected

One thing I've noticed is that xAP Jabber seems to get disconnected from the Y! network after not sending a message for a period of time. I fired up the xAP message viewer to see if there were any messages being sent when Jabber disconnected. I found that Jabber would send a User.Status message with the field xstatus set to nothing (as opposed to "Online" or "Logged off").

I wrote a small raw script for xAP Floorplan to send a message whenever it detects that status message:
Sub Main()
status=xap.getfromraw(rawmessage, "User.Status,xstatus")
user=xap.getfromraw(rawmessage, "User.Status,username")
' replace w/ your user name
if user= "yahoo_house_xyz@yahoo.localhost" then
' save status in global in case you want to use it somewhere else
SetGlobal "IMHouse", status
if Len(status) < 2 then
xAPBody="line1=wake myself up ping"
' replace w/ your user name
xAPTarget="mi4.jabber.jabber:yahoo_house_xyz.yahoo.localhost"
' the following 3 lines need to be all 1 line
x=xap.SendxAP("display.text" & chr(10) & "{" & chr(10) &
xAPBody & chr(10) & "priority=1" & chr(10) & "}" &
chr(10),"message.display",xAPTarget)
end if
end if
end sub

I saved it as jabberstatus.txt. The next step was to trigger this every time xstatus changed. Following this Floorplan example, I created a raw device for Jabber in floorplan. I filled in the fields as follows:
  • Alias = jabber status
  • Source = mi4.jabber.jabber
  • Class = Messenger.Event
  • Section = User.Status
  • Item = xstatus
  • Script to run on update = jabberstatus
That's it. Now, whenever Floorplan detects Jabber being disconnected, it sends a message that causes Jabber to reconnect. So far, it seems to work. Jabber has stayed connected overnight for the first time since I installed it. Use the xAP Message Viewer to resend the User.Status messages to test out the script. Make sure you check the Script Debug tab in Floorplan to make sure you didn't make any typos (or copy any of mine!)

Tuesday, October 24, 2006

HAL Deluxe

I was one of the lucky people who got a copy of HAL Deluxe for about $7 back when Lowe's was blowing them out.

While HAL has an exceptional voice recognition system, it has such a kludgy interface. It doesn't support any kind of scripting engine eventhough every major and just about every minor HA package does. It has a tedious point & click interface to add events, and there's no way to easily copy/paste events to make things more efficient. For whatever reason, they don't support MS standard SAPI like just about every other HA package. You have to buy their HAL Voices to get decent speech synthesis. If you have AT&T Natural Voices, you're out of luck. You still have to buy HAL Voices eventhough they are derived from the AT&T voices!! Fortunately, I won a gift certificate from Automated Outlet during a Cocoontech chat, so that lessened the pain of having to buy a copy of something I essentially already had.

Other than that, it does OK for what I use it for. ;) In the time since I bought HAL Deluxe, they've come out with something called HALi, an ActiveX programming interface to HAL. Recently, I finally gave in and upgraded to HAL 3.5 just to get HALi and see if it could be useful for anything. I'm hoping HALi will provide a better way to integrate HAL with starCOM, but 2+ years after its release, there were no samples of HALi using Windows scripting...

Using xAP Jabber with Yahoo! Messenger

xAP Jabber is another app from mi4.biz. It can send/receive instant messages to/from your IM client. Its native protocol is of course Jabber, but there are many Jabber servers that have transports to communicate with other IM platforms, like AIM, Yahoo and MSN. I use Yahoo! Messenger and this is how I set up xAP Jabber to talk to my IM client.

First thing I needed to do was to get a Jabber client so I could create a Jabber ID. I downloaded Exodus. Next, I needed to find a Jabber server with the Y! transport, so I looked over here. After clicking on a server and finding it supported Y!, I fired up Exodus and created a Jabber ID (I'll call it jabber_house_xyz). Now, this Jabber ID needs to have its own Y! ID to communicate with Y! users. So I created a Y! account (yahoo_house_xyz) for the Jabber ID to link to. I had to add it to the contact list of my real Y! ID (I'll call that ID my_real_idxyz) and then add my_real_idxyz to the contact list for yahoo_house_xyz. Next I needed to get the Y! transport up, so I followed this (except selecting Y!). I registered it with my new Y! id (yahoo_house_xyz).

After that, I added my_real_idxyz to the Exodus contact list, selecting Yahoo for contact type and entering the ID. Now, my Jabber account was set up and linked to Y! After exiting Exodus, I started xAP Jabber which brought up the settings file. I entered the login info for my Jabber account then followed the instructions to set up the rest of the app.

One thing I needed to do to get it to work with Y! was to configure xAP Jabber to send messages as chat messages instead of headline messages. (sending #set displaystyle yes) to the xAP client. For some reason, it dislikes headline messages.

Now, xAP Jabber is all set up to forward Message.Display messages to my IM client. It's also possible to issue commands over IM to my house to control things, etc. Each IM sent to xAP Jabber gets sent as a xAP message, which can be acted upon by a scripting engine (like xAP Floorplan). Actions can be performed and replies can be sent based on the commands xAP Jabber received.

Monday, October 23, 2006

xAP Switchboard

This is an app I'm playing with at the moment. You can get it from mi4.biz. Using the Outlook Plugin from mi4.biz, Switchboard imports your contacts from Microsoft Outlook 2003. When it receives caller ID info, it will do a lookup in your Outlook database to identify the caller. It can then send this info to OSD (on screen display) or TTS (text to speech) devices/apps.

I have Outlook 2002, but I just found out that the Switchboard database is MS Access format. So from Outlook, select File > Import and Export > Export a File > Microsoft Access > Contacts. Then select a file name & click Next to create the Access database of the Outlook contacts. Next, run Access, open TelDir.mdb in the Switchboard directory. Then select File > Get External Data > Import and select the file you created from Outlook. You'll get a popup call Import Objects. Select Contacts and click OK. Now in the TelDir: Database window, double click Contacts which opens up your Outlook database. Select the first column, right click and select Insert Column. Adding this blank column helps match the Outlook database with the Switchboard database for an easy copy & paste. Now just highlight all rows with the data you want, right click, select Copy, open the XAP_Contacts database, scroll to the bottom, select the very last row, right click and paste in your contacts. Close the Contacts window. Now you can delete the Contacts database from the TelDir: Database window. Exit and that's it. You should have your Outlook contacts available.

Unfortunately, this doesn't lend itself to updating frequently, but at least it's an easier way to get the info into Switchboard if you don't have a version of Outlook other than 2003.

xAP

One of the many parts of my HA system is xAP. If you don't know what it is, there's a great write up on xAP over on Cocoontech. I've been playing around with numerous xAP applications lately and I'll talk about some of these later.

Two of my favorite xAP sites are:
mi4.biz
www.xapautomation.org

and there's also a xAP group over on yahoo groups.

Links to some random freeware apps I use

sysinternals - a bunch of very useful freeware utilities
truecrypt - set up an encrypted partition on your disk, great for usb keys
mrtg - tool to chart your network traffic/utilization, is someone leeching off your connection?
rrdtool - another excellent graphing app. I use this for charting temps, disk/cpu utilization
xming - lightweight xwindows server for windows
cygwin - linux-like commands/apps for windows
webcam 2000 - camera server. I use this for my security cams
realvnc - control remote computers
blat - command line mail sender, I use this for sending sms or email w/ status updates from the house
ms sapi 5.1 - microsoft speech synthesis engine

Sunday, October 22, 2006

ASP page for TC+ Flags

One of the first things I added to my system with sC was the ability to replicate all the WinEVM MegaController functions in ASP scripts (using the IIS web server in Win2k). This code enumerates all the flags defined in WinEVM, shows their state and allows them to be turned on & off. It can be easily modifed for relays (replacing DeviceCollection = Devices.Flags; with DeviceCollection = Devices.Relays;) or X10 (DeviceCollection = Devices.X10Devices;). You can find my first iteration of ASP pages over on the PTS example page. Grab the ASP MegaController zip file for sample code to control IR, DIs, AIs, Timers and Variables from your browser.


<%@ LANGUAGE=JScript %>
<html>
<head><title>Flags</title></head>
<body bgcolor=#A5DFBA><center><font face=arial>
<%

// flag page
// set/clear flags
// (c) 2002-2006 kJ's Doghouse.com
// http://www.doghouselabs.com

var Devices;
var DeviceCollection;
var EnumDevice;
var EnumItem;

//
Devices = Server.CreateObject("starCOMplus.PTSDevices");

var Name = Request.Form("Name").item;
var Control = Request.Form("Control").item;

// perform the command if we got one
if (Control=="ON") {
Response.Write("<B><center>Issued: " + Name + " ON</center></b>");
Devices.Flags.Item(Name).State = 1;
} else if (Control=="OFF") {
Response.Write("<b><center>Issued: " + Name + " OFF</center></b>");
Devices.Flags.Item(Name).State = 0;
} else {
Response.Write("<B><center> </center></b>");
}


Response.Write("<center><table border=0>");
var odd=1;

// enumerate flag devices
DeviceCollection = Devices.Flags;
EnumDevice = new Enumerator(DeviceCollection);
for (; !EnumDevice.atEnd(); EnumDevice.moveNext()) {
EnumItem=EnumDevice.item();
if (EnumItem.Name.length >1) {
if (odd==1) {
Response.Write("<Tr>");
}
Response.Write("<td>");
Response.write("<b>" + EnumItem.Name + "</b></td>");
Response.Write("<form method=POST action="+Request.ServerVariables("Script_name")+">");
Response.Write("<td width=1> </td>");
if (Name==EnumItem.Name) {
if (Control=="OFF") {
Response.Write("<td width=55>\n");
Response.Write("<center><font color=red><b>");
Response.Write("OFF");
} else if (Control=="ON") {
Response.Write("<td width=55>\n");
Response.Write("<center><font color=green><b>");
Response.Write("ON");
}
} else {
if (Devices.Flags.Item(EnumItem.Name).State==0) {
Response.Write("<td width=55>\n");
Response.Write("<center><font color=red><b>");
Response.Write("OFF");
} else {
Response.Write("<td width=55>\n");
Response.Write("<center><font color=green><b>");
Response.Write("ON");
}
}
Response.Write("</td>\n");


//ON Button
Response.Write("<td>");
Response.Write("<input type=submit value=ON name=Control>");
Response.Write("<input type=hidden name=Name value=" + EnumItem.Name + "></td>");
//OFF button
Response.Write("<td><input type=submit value=OFF name=Control></td>\n");
Response.Write("</form>");

// blank form to keep spacing good
Response.Write(" <form method=POST action="+Request.ServerVariables("Script_name")+">");
Response.Write("<td width=28> </td></form>");

if (odd<2) {
Response.Write("<td width=1> </td>");
odd=odd+1;
} else {
Response.Write("</tr>");
odd=1;
}
}
}
Response.Write("</table>");



// clear stuff out
Devices = null;
DeviceCollection = null;
EnumItem = null;
EnumDevice = null;

%>
</body></html>

starCOM

The original software provided by JDS (WinEVM - docs here) never evolved past a Win 3.0 look and feel. It had absolutely no provisions for any kind of networked interface. Because of these limitations I started using Pine Tree Systems' starCOMPlus (sC). I still use WinEVM, but only for downloading schedule updates into the TC+. Otherwise, sC is connected full-time.

I got involved with sC during its early alpha releases (2001ish I think) and I was heavily involved in hammering on it throughout its beta period. Because of that, sC is the most complete and STABLE interface for the JDS line of controllers. :-D

sC exposes all the devices and events of the TC+ to a COM/ActiveX interface allowing the TC+ to be controlled by other external apps. This allows the TC+ to be networked and greatly improves its usability in a home automation system. sC also provides much more flexible and powerful programming, unlike the rigid, limited "starglish" language in WinEVM. You can create your own WinEVM-like schedule (called a hosted script), that runs inside sC. It doesn't replace the WinEVM schedule, but runs on the HA server in parallel. Programming sC can be done with VBScript, JScript (my preference) or probably even Perlscript (which I've never tried but probably should).

All mission critical apps are in the WinEVM schedule that is downloaded directly into the TC+. I'm not trusting HVAC and irrigation control to a Windows 2000 PC, although my server has been extremely stable considering the amount of stuff running on it & all the tinkering I do on it.

Saturday, October 21, 2006

Hardware pieces

My system is a combination of a hardware controller and software. The hardware controller is a JDS TimeCommander+ (TC+). It's been discontinued by JDS so you won't find much information on it on their site anymore. It features 2-way X10, analog inputs (which I use for temperature sensors), digital inputs (for contact closure detection) and relays (to control the HVAC system).

The TC+ is connected to an InfraRed Xpander which provides 2-way infrared control. However, the IR receiving functionality is not very reliable. I use a different method using Slim Devices SliMP3s (discontinued, noticing a pattern?) as IR receivers.

The TC+ is in the wiring closet connected to a serial port on the home automation server about 12 feet away. The serial connection is over a Cat-5 cable with DB9-RJ45 connectors. The HA server is a 2.6 Ghz Celeron running Win2k Pro and loaded with 1 GB RAM, about 700 GB of disk space, a DVD burner, 2 modems and 2 video input cards. I'll add more details later...

Ok here goes...

This is going to be my attempt to document what I've been doing with my home automation system ( have a look http://www.doghouselabs.com/autohouse). It's been a hobby of mine for about 10 years or so, but I'm continually adding stuff. Hopefully this blog will capture some of what I'm doing now. I plan for it to have a lot of detail and code samples. And when I'm not working on something new, I'll try to talk about what I've already built into the system. So be prepared for flashbacks sprinkled throughout the story line. It works for Lost so maybe it'll work for me ;)