Quick Post - xPLGVoice Preview

Here's a quick screenshot of my new xPLGVoice app. It allows control of some aspects of Google Voice via xPL messages - specifically, sending SMS, initiating calls and enabling/disabling your forwarding numbers. I hope to release it soon.

Bridging xPL Revisited

I originally talked about bridging xPL across the Internet using EventGhost in this post. More recently, I wrote about pulling the sender/receiver plugins out of EventGhost here. I did that but haven't gotten around to cleaning up the code. The problem with either of solution is that it's not very secure, so I've always tunneled them through SSH. If either Internet connection goes down, I have to manually restart the SSH session (since leaving my password and keys lying around so I can automate the login probably isn't a good idea).

Instead, I've decided to bridge the connection using IM with our Blabber app. I just set up 2 GMail accounts, one for each house, and added one to the other's chat list and fired up the local and remote Blabber apps. I wrote a script to capture certain xPL messages and embed them in an xPL message sent to Blabber. On the receiving end, the xPL message generated by Blabber is parsed out in my xPLSCU scripting engine.

The big advantage is there's no need to manually set up SSH tunnels as the messages go over Google's chat server encrypted with TLS. If the link goes down, Blabber will just keep trying to send messages until the link comes back up. No manual intervention necessary. I've had it running for a day now and it seems to be working out well. I've already had the remote ISP connection go down for about an hour, and when the cable modem was automatically restarted, the Blabber conversations continued on their own. Very cool! For added security, you can also encode your messages before sending them to Blabber.

Jobless Still...But "Work" Continues

The deal with the early stage startup (which recently closed a Series A round from 2 firms) fell through. It left a bad taste in my mouth. They dragged the whole process out over a week only to email me this: "Thanks for taking the time to speak to us. Unfortunately, your background is not a good fit for the current position we have open." It's odd they came to this conclusion after I nailed the technical screenings with their engineers, and they thought I was good enough to talk with the VP of Engineering and the CTO. They even asked for my references, but in the ensuing 5 days, they never called a single one. I also had an inside person there who said everyone was impressed by me and it was pretty much a done deal. However, their Director of ASICs admitted to having difficulty with the compensation package given where I was at. After doing a little research on my own, it looked like I would be offered a 25% cut. So, to save face and not be called cheap, they chose to say my background was not a good fit rather than saying they wasted everyone's time because they can't afford me. What a joke.

Instead of long hours at the early stage startup with a menial salary, I've accepted a consulting gig at a large, public company. I won't have to cut back on my gadget spending and I'll only work 40 hour weeks, leaving plenty of time for my own projects.

On that note, my Google Latitude project is progressing nicely. Still, not yet at the point to post screencaps, etc. Another project I started yesterday was creating an xPL bridge in Python. I had already created a rudimentary way to link xPL networks across the Internet using EventGhost and the Network Sender & Receiver plugins. I'm still using EG at our other house, but I don't really need to use EG on my HA server, so I stripped out the Network Sender & Receiver plugins and made them into standalone Python scripts. I've also gone one step further and made the receiver script capable of re-sending the bridged xPL messages as if they were sent by the original source. Now, I can fully view the remote xPL network and configure the remote apps as if they were local. There's still a lot of tweaking and clean up to do, and I am pondering building a C# xPL bridge just to make things less of a hack. The thing is, I still use EG on the remote network and the thin client it runs on is rather limited, so I don't really want to add another app on that machine.

xPLBattWatch - A Battery Charge Monitor

This is the first release of xPLBattWatch. This app is used to monitor the battery level and charge status of a laptop. It issues sensor.basic messages based on changes in the charge level and status. It may be a little rough, but I thought I'd throw this out there and let you play with it. More details and download here.

xPLWebControl Released

I released an xPL interface for CAI's WebControl. The app communicates with the WebControl via its web interface since it has no other means of control. You can read more about it and download it here.

xPLWav Enables Remote Controlled WAV Playing

I've released another app, xPLWav, which allows you to make a remote computer play a WAV file. I'm using this in combination with our Noise app to automatically scold our dog when he's barking and we're not home :) This app is necessary since the closet PC to where our dog usually barks (and where Noise is installed) is not the HA server. To do this, I've recorded WAV files of us saying corrections ("Quiet!", "Stop that!", etc.) and upon a short period of no noise, an automatic praise is played ("Good dog!", etc.) So far it's working. Anyway, give it a try.

Noise 1.1 A Complete Rewrite

I wasn't really happy with the XAML based GUI of my original Noise app. It required .NET 3.5 which limited what machines it could run on. I didn't like the configuration method I chose either. So, I completely rewrote it. Same link as before.

xPLGPix connects xPL to Google's Picasa Web Albums

Continuing with my Google tangent, I've just created a new app called xPLGPix. If you send it an xPL message specifying an album and JPG file, it will upload the picture to the specified album on Google's Picasa. You can read more about it here. I've tested it just a little, so it may be a little rough around the edges.

xPLGSheet: An xPL interface to Google Doc Spreadsheets

I'm welcoming in 2010 with a new app called xPLGSheet. It allows you to query and edit a GDocs spreadsheet using xPL messages. Aside from the typical use of automated updates of a database, this app opens up possibilities of syncing remote systems via a shared spreadsheet. For more, go here.

New App: Noise

No, I haven't fallen off the face of the earth, but I admit that it's been a while since I last posted. I've been taking a break from automation stuff over the holidays, but I did work on something I've always wanted to implement...

Have you ever wanted to be automatically told that the baby is crying or the dog's barking? Introducing my latest xPL app, Noise. It uses your PC's mic (or possibly other input device), samples the input for a period of time, and issues an xPL sensor.basic message with information about the noise it detected, if any. You can read more about it here.

xPL Tweeting

I'm not sure why this hasn't been done earlier, but it looks like I'm the first to integrate xPL and Twitter. It was really easy to do and only took a few hours. A lot of the time was just figuring out what API to use and investigating OAUTH. I ended up using twitterizer's API, which doesn't support OAUTH. That's fine because I want this app to run standalone without any user interaction required for login, which OAUTH requires. The downside is the tweet source will show Twitterizer and not xPLTweet. Twitter no longer takes registrations for apps that do not use OAUTH, so there's no free publicity for my app in your Twitter streams. Bummer.

xPLTweet grabs OSD.BASIC messages explicitly targeting it (ie doghouse-xpltweet.yourinstance). It grabs the text portion of the message and tweets that. That's pretty much all it does. A future enhancement I may do will be to monitor who you're following and issue OSD.BASIC messages when they update their streams, but it could be messy if you follow many active Twitterers.

In any case, you can grap xPLTweet here.

Back to Juggling Multiple Projects

Earlier this week, I started work on a new xPL app, currently called xPLSys. It monitors disk space, memory and cpu utilization and sends xPL messages containing the data. I'm not exactly sure how I want to package those messages as there really isn't a schema to hold the info I want to send - so either I make up my own schema or send multiple messages - one for each stat being watched. It's been running for a few days now but has been pushed aside for something else.



My Brultech ECM-1240 power monitor arrived Friday, and I started installing it that evening. I finished up most of the wiring Saturday morning and have been deciphering the serial protocol for real-time updates. I've been using our xPLSerial app to interface to it and code up the interactions with the built in scripting engine. I have to say, it's a very cool app and you should download it and donate ;). Despite being distracted by CAL's upset of Stanford (HOORAY), I figured out the serial protocol last night and have now been fine tuning my script. I've been thinking about rolling this into an app, but there is virtually no cpu utilization with the script running in xPLSerial.



I did make a change to xPLSerial to assist with debugging and that change will get pushed out eventually. I'm also considering removing the xPL from the name and calling it something else. It's really a serial port scripting engine with built in xPL functionality, but doesn't require xPL. In the meantime, I need to figure out what I'm going to do with the power data I'm getting every second.

xPLSerial Released

It's been a while since I released an app. xPLSerial gloms together a generic serial port interface, xPL and a scripting engine. Read more about it and download it here.

xPLSerial

I've decided I needed to code up something to deal with serial port devices I may want to interface with. I've put serial device handling into my starCOMUltra hosted script as well as in my sCU-xPL bridge (xPL sCU), but while I'm debugging stuff, I don't want to keep reloading those apps as much of the HA system relies on it functioning correctly. There doesn't appear to be an xPL app that allows generic serial port interfacing, so I figured this would be a good (and easy) app to write. A majority of the code comes out of xPL sCU since it already does serial & xPL as part of its features. Because of this, I've already got a prototype running. It relies on an internal scripting engine to set up the serial port, send data to it and parse data from it. You can then issue xPL messages as necessary. Right now, I'm working on interfacing to an Ocelot I just acquired.

My Take on SageTV Server Power Savings

I've been working off & on with the media server's power management. Since we have a DirecTiVo, most of our live viewing is done on that along with some recording, so the Sage server doesn't need to be online unless we're watching some recorded content from it or it's recording. So I can really keep this thing powered down 70% of the time, and almost 90% of the time since it's summer. So I spent some more time figuring out when SageTV is being used and came up with this write up. My solution, like our home automation system, integrates a lot of separate pieces to get things just right.

Let's get to it. First off, as in my previous post, we use the exported sage schedule. Grab this Perl file and save it as a .pl file. The script is fairly simple. It takes a command line argument, the path to the schedule file, and figures out what hours of the day SageTV is recording. It puts that info into a mySQL database called mystuff with username and password as the login credentials. You don't have to use mySQL, that's just what I use to pass data between different components of our system.

The next step involves our xPL Plugin for SageTV and your favorite xPL scripting engine. The engine we use is our homebrew xPL sCU - an xPL bridge to starCOMUltra, the HA software we use. With the xPL plugin for Sage, you can query it for which clients & extenders are currently playing media (or you can rely on the hidden feature where the plugin will check the playing status whenever it receives any xPL message - it will send an update only if the status has changed). In xPL sCU, I've scripted up logic for processing those updates:

function xPLReceived(e,schema,source,dest,msgtype) {
    switch(schema.toLowerCase()) {
        case "doghouse-sagetv.media":
            if (schema.toLowerCase()=="media.mptrnspt") {
                mpName=e.XplMsg.GetParam(1, "mp");
                mpCommand=e.XplMsg.GetParam(1, "command");
                mpStatus = mySQLQuery("locals","value","SagePlayerStatus");
                switch (mpName) {
                    case "LR":
                        mpID=0;
                        break;
                    case "FR":
                        mpID=1;
                        break;
                    case "Master":
                        mpID=2;
                        break;
                }
                if (mpCommand=="play") {
                    mpStatus=mpStatus | Math.pow(2,mpID);
                } else {
                    mpStatus=mpStatus & (7-Math.pow(2,mpID));
                    if (mpStatus==0) {
                        sTimers.load("SageClientIdleTimer",20*60);
                    }
                }
                if (mpStatus!=0) {
                    sCUDevices.Flags.Item("SageClientIdle").State=0;
                }
            }

What the snippet does is process xPL messages about the media players' state. If any player is in use, it clears the SageClientIdle flag in starCOMUltra. If a player becomes idle, it checks to see if all the players are idle and starts a 20 minute timer. Elsewhere in the code (not shown) is if that timer expires and all players are still idle, the SageClientIdle flag is set. You can choose your own idle timer length.

Now we have the player and recording status, but wait, what if I'm using the server for some other purpose? The only way I'd be using the server is via remote desktop, so i set up an EventGhost macro on the Sage server to send xPL messages when an RDP session is initiated or closed. A timeout period after a session is closed will determine if that aspect of the server is idle.

Finally, we have all the pieces in the place. In starCOMUltra, where everything comes together, we have this piece of code that runs hourly (along with the parseSage.pl script):

        // to prevent unneeded throttling, we'll check if sage is currently
        // recording or will need to in the next 3 hours
        sageRecord=0;
        for (i=0;i<4;i++) {
            if (mySQLQuery2("sagesched","value","hour",TimeHour()+i)==1) {
                sageRecord=1;
            }
        }
        // check if media server is awake and no RDP sessions open and no recordings
        // for the next 3 hours and no players in use - then 10 after the hour
        // we'll put the server to sleep
        if (mySQLQuery("locals","value","MediaAwake")==1 && \
            mySQLQuery("locals","value","mediaRDP")==0 && \
            Devices.Flags.Item("SageClientIdle").State==1) { \
            sageRecord==0 && mins==10) { 
            LogDevice("#        stop sage server");
            HibernateMedia();
        }

To wake up the server manually, all our viewing areas have a 3Com Audrey touchscreen nearby. It has a screen to wake up the media server which takes a few minutes. Not perfect, but like I said, we use the DirecTivo for most of the on demand TV watching. Plus, most of the time, we watch recordings during prime time and the server is already on.

"That's all!" ;) Obviously, this is very specific to our setup and situation, but hopefully there are ideas and code bits that may be useful to you.

Remote W800 Deployed & Remote System Summary

I just installed the new W800 at our other house. In keeping with my tradition of cheapness, I ordered the bare $49 W800 instead of the $75 kit. I made my own power adapter from a leftover 9V wall wart and a 3.5mm mono jack (tip positive). I had 3 different 9V wall warts to choose from in my junk box, one output 8.7V, one 10.5V, and the third put out 11.5V. The W800 would only work with the highest output wall wart, which jives with the output from power supply from the first W800. Instead of the kit antenna, I connected the W800 to a set of leftover TV rabbit ears. I'm not looking for long distance reception as it's a small place. Finally, I had an extra serial cable in a spare parts bin.

I'm using xPLW800 and it works perfectly. I installed a few devices from my leftovers - 2 DS10A's as leak detectors and a few Hawkeye motion detectors. The motion sensors, combined with the Bluetooth tracking, will allow the system to track occupancy and execute the security lighting scheme if we're not around.

So far, the remote thin client is loaded up with:
* CM11A on a USB-serial adapter, using xPLCM11 for lighting/device control
* W800 on COM1, using xPLW800 for occupancy, dusk/dawn & leak detectors
* xPL Rocket for remote pan/tilt control of the USB rocket launcher & AirLink netcam wire-tied to it
* BlueTracker for occupancy detection
* EventGhost with xPL Plugin for all automation logic

With those apps running, there's still about 190MB of RAM free after I did the 512MB upgrade. CPU utilization is low, typically around 5-10% and briefly jumping to about 20-25% when an X10 command is issued. I can probably add a couple more small apps without killing the system.

The thin client is turning out to be the perfect device for our application. It is slow, but we don't interact with it at all, unlike our main home's automation server. It has low power consumption (10-12W), solid state disk reliablity and auto-powers on after an outage. I thought about putting it on a UPS, but since we won't be at house very often, I'm wary of leaving a UPS there. I've had one UPS burn up after several months and 2 others ended up with cracked and leaking batteries.

I've been very pleased with EventGhost as the heart of the system. It's lightweight in memory and CPU, has many useful plugins (Network Sender & Receiver are key for me) and has tons of flexibility with integrated Python scripting. It wouldn't be very useful to me if it weren't for the great xPL plugin I wrote ;)

Linking xPL Between Two Homes Over The Internet

As I wrote before, I would like to link xPL messages from our 2nd home. I needed to figure out a way to wrap the UDP xPL messages in TCP and tunnel them through some kind of connection. I decide to use EventGhost's Network Event Sender/Receiver plugins so I wouldn't have to write a separate app.

The first step is to establish a secure connection between the two networks. I chose SSH as I already have a device capable of being an SSH server at our 2nd/remote home. I will establish the SSH connection using Putty from our 1st home.



I'm creating a tunnel that will map port 8888 on the remote SSH server to port 8888 on my local machine. I'll need to click the Add button to enable it. I also checked Remote ports do the same, which will allow devices on the remote network to connect to the remote SSH server's port 8888. To enable this remote forwarding, you may have to configure your SSH server. For example, with OpenSSH, you need to add GatewayPorts yes to the sshd_config file.

Once that's set up, I can establish the SSH connection. On the local machine, I will add the Network Event Receiver plugin, point it to port 8888, set a password and an event prefix. On the remote machine, I will add the Network Event Sender plugin and point it the remote SSH server's port 888 and set a password to match the receiver.


Local EventGhost

Remote EventGhost

Now, in the remote EventGhost, I create a macro that captures all xPL messages and sends them the Network Event Sender.



Finally, on the local EventGhost, I create a Network Event Receiver macro that parses the data received and issues an xPL message with the original contents, but with the local EG as the sender instead of the original sender.




That's all there is to it. Now, I've got remote xPL messages coming across to my home network. You can easily add a remote EG receiver and local EG sender to allow xPL messages to flow in the opposite direction.

xPL Rocket Launched

I've released my xPL hack to the SharpLauncher software. Now, you can have remote, network control over your arsenal of USB rocket launchers. You can get it here. Strap a webcam on top of your rocket launcher and you have a super cheap pan & tilt camera.

Remote Thin Client

I have been working on thin client #2 which will be deployed at our other home. One thing I wanted to get running on it was bluetooth so it can track our presence with BlueTracker. It was quite a pain to figure out since XP Embedded doesn't really support bluetooth so I had to copy drivers, DLL's, etc over from an XP system. In addition to the BT radio drivers, it also needed drivers for the BT enumerator, BT PAN and BT RFCOMM protocol. Finally, I also needed to copy over the file irprops.cpl, which the 32feet.NET BT library needs. Figuring that out wasn't easy as BlueTracker would fatal with this error:

EventType : clr20r3 P1 : BlueTracker.exe P2 : 1.0.0.0 P3 : 4994a8a1
P4 : inthehand.net.personal P5 : 2.3.0.0 P6 : 483fc5c5 P7 : 15d
P8 : 8 P9 : system.dllnotfoundexception

After some Googling, I found this reference that points to an attribute called DllImportAttribute. I opened up InTheHand.Net.Personal.dll in my trusty Emacs editor, searched for that string and found irprops.cpl. Once that was copied over, BlueTracker worked.

Now, I need to come up with a mechanism for getting BlueTracker status across the Internet to our home. Since xPL is UDP, I can't just tunnel it through an SSH connection.

xPL Plugin for SageTV released

I have released my first rev of the xPL plugin for SageTV. You can read more & download it here.