EventGhost Disappearing From Our System

Given my latest effort to bridge our remote network via IM, I've started scaling back our usage of EventGhost on the remote thin client and our HA server. This weekend, I started converting the automation code on the remote thin client over to xScript. I've had it running overnight and it's been working so well I've shutdown EG. It hasn't been running on the HA server for a few weeks since I extracted the network sender/receiver plugins to Python scripts. The server actually seems more stable. I've always suspected EG of having memory leaks that force me to reboot the HA server and remote thin client periodically. At one point, I had 3 machines running EG and now I don't have any. EG is also facing an uncertain future as the developer is planning to stop work on it, and now seems like a good time to transition away. This also means I will probably stop maintaining the xPL plugin. There aren't any outstanding problems with it and I haven't touched it in over a year, so it's not a big deal.

Bridging xPL Revisited

I originally talked about bridging xPL across the Internet using EventGhost in this post. More recently, I wrote about pulling the sender/receiver plugins out of EventGhost here. I did that but haven't gotten around to cleaning up the code. The problem with either of solution is that it's not very secure, so I've always tunneled them through SSH. If either Internet connection goes down, I have to manually restart the SSH session (since leaving my password and keys lying around so I can automate the login probably isn't a good idea).

Instead, I've decided to bridge the connection using IM with our Blabber app. I just set up 2 GMail accounts, one for each house, and added one to the other's chat list and fired up the local and remote Blabber apps. I wrote a script to capture certain xPL messages and embed them in an xPL message sent to Blabber. On the receiving end, the xPL message generated by Blabber is parsed out in my xPLSCU scripting engine.

The big advantage is there's no need to manually set up SSH tunnels as the messages go over Google's chat server encrypted with TLS. If the link goes down, Blabber will just keep trying to send messages until the link comes back up. No manual intervention necessary. I've had it running for a day now and it seems to be working out well. I've already had the remote ISP connection go down for about an hour, and when the cable modem was automatically restarted, the Blabber conversations continued on their own. Very cool! For added security, you can also encode your messages before sending them to Blabber.

More Tracking

I continue to tweak the map resulting from the tracking info (see my updated video). I implemented a suggestion in this thread to grab the "address" corresponding to the longitude/latitude of a sample. That now shows up as the location when mousing over a marker. The mouseover kicks off an AJAX fetch of the geolocation info from Google Maps, which processes the JSON data and fills the popup. Clicking the marker brings updates the popup to show 3 nearby items and clicking it again brings up the original data.

A major change is I'm now using the Google Maps direction service and renderer to do the actual distance calculations between two markers and to draw the line connecting them. This ends up being a lot more accurate than my original straight line calculation and, of course, the line actually follows the road traveled. The more accurate distance calculation means a better speed estimate for that interval as well.

Right now, the code is a disaster, but I plan on putting it on my website after I clean it up.

Google Latitude Tracking Live

Today, I've fully deployed our tracking scheme using Google Latitude. My wife and kids have started their road trip and I'm tracking their progress. Here's how things look so far.

Using the Latitude data, I can determine their position in the past 2 minutes (it appears to update only every 2 minutes). Using two points and their timestamps, I can estimate the approximate speed they were driving between the two points. As I mentioned in a previous post, I'm storing the locations in a MySQL table and through AJAX, I'm able to update a map with their path and show information bubbles over each point with some information, which is what you see in the video linked above. You can see an occasional large gap between points as they move in & out of 3G coverage, but it's not too bad since we don't use AT&T. HA!

Jobless Still...But "Work" Continues

The deal with the early stage startup (which recently closed a Series A round from 2 firms) fell through. It left a bad taste in my mouth. They dragged the whole process out over a week only to email me this: "Thanks for taking the time to speak to us. Unfortunately, your background is not a good fit for the current position we have open." It's odd they came to this conclusion after I nailed the technical screenings with their engineers, and they thought I was good enough to talk with the VP of Engineering and the CTO. They even asked for my references, but in the ensuing 5 days, they never called a single one. I also had an inside person there who said everyone was impressed by me and it was pretty much a done deal. However, their Director of ASICs admitted to having difficulty with the compensation package given where I was at. After doing a little research on my own, it looked like I would be offered a 25% cut. So, to save face and not be called cheap, they chose to say my background was not a good fit rather than saying they wasted everyone's time because they can't afford me. What a joke.

Instead of long hours at the early stage startup with a menial salary, I've accepted a consulting gig at a large, public company. I won't have to cut back on my gadget spending and I'll only work 40 hour weeks, leaving plenty of time for my own projects.

On that note, my Google Latitude project is progressing nicely. Still, not yet at the point to post screencaps, etc. Another project I started yesterday was creating an xPL bridge in Python. I had already created a rudimentary way to link xPL networks across the Internet using EventGhost and the Network Sender & Receiver plugins. I'm still using EG at our other house, but I don't really need to use EG on my HA server, so I stripped out the Network Sender & Receiver plugins and made them into standalone Python scripts. I've also gone one step further and made the receiver script capable of re-sending the bridged xPL messages as if they were sent by the original source. Now, I can fully view the remote xPL network and configure the remote apps as if they were local. There's still a lot of tweaking and clean up to do, and I am pondering building a C# xPL bridge just to make things less of a hack. The thing is, I still use EG on the remote network and the thin client it runs on is rather limited, so I don't really want to add another app on that machine.

August Already?? and Google Latitiude

I was figuring I'd spend a lot of time on the house and the automation system while I was job hunting, but it didn't turn out that way. It's been about 6 weeks and I really haven't done a whole lot. I did finish up most of the trim for the flooring, but I had to stop and find some flexible quarter-round to go along the bottom of our staircase (which has a full 180 degree curve on one side). I did pick some up from Trimster, but I haven't thought much about how to stain it, so it's sitting around.

As for automation stuff, I've been revamping our AJAX based floorplan GUI. Given all my free time, I've been reading up on CSS3 and jQuery and have completely re-written large portions of the code. My HTML page used to have markup, JavaScript and CSS all mixed together, but now I've gotten everything updated to typical coding standards, with the HTML page containing only the markup, and the CSS and .js files have grown significantly. It's definitely much neater to read now and should make maintaining and updating it easier as well. I figure at some point, I will migrate it to HTML5 and that will be another major overhaul, but for now, I'm very pleased with the new CSS and jQuery things I've learned.

When Google Latitude first came out, I put a placeholder at the bottom here, but had so many things going on, I forgot all about it. This thread reminded me about Latitude tracking and with the wife planning a 5-day roadtrip with a girlfriend, I finally decided to get off my butt and implement some tracking (with her permission of course! :)

First off, we need to get a public badge for Latitude - you can find out more here. We already have Google Maps on our cells so we just need to enable Latitude on the app. Next, we need an automatic trigger to start Latitude tracking. We use RFID tags in the cars and bluetooth on our phones for tracking, so when someone's phone and a car are gone, we can start the Latitude tracker.

The tracker is just a perl script that periodically fetches the Latitude badge's JSON feed and parses its information (latitude, longitude, timestamp, reverse geocode info). That information is stored in a MySQL table that is reset for every new trip. Finally, I've written an AJAX page that fetches the location data from MySQL via PHP and shows each data point on Google Maps with a line connecting the points. I plan to post more details as I debug this new feature.

Finally, I am about to close a deal to join an early stage startup. The company is in a very promising space and is at the right stage for me to be able to contribute a great deal to the product development. However, early stage equals long hours, but I'm ready for another startup. HA stuff will just have to be put on the backburner...

Charmed Quark Controller Going Open Source

Dean Roddey, the creator of CQC, has thrown in the towel, but is planning on releasing CQC as open source. It's very unfortunate as CQC is an incredibly powerful (and incredibly complex) software package. I am looking forward to checking it out for building touchscreen interfaces, but I really like the platform independence of my AJAX GUI. I will, however, try to add xPL support to CQC. In any case, it makes you wonder how the makers of HAL can possibly survive.

Update: The link above has been updated by Dean. He is no longer planning on open sourcing CQC...

I Am Still Here

No, I have not fallen off the face of the earth. I've been working on some upgrades, but nothing related to automation. I've been replacing our downstairs carpet with laminate flooring, and while that's been going on, the wife & kids have been helping me repainting too. I know, there are hardwood snobs out there who think laminate flooring is like linoleum. Granted it's not hardwood, but we already have engineered hardwood in the kitchen & nook areas, and it's been gouged to death by our previous and current dog. I'm not going to pay $20,000 to have the rest of the house done in hardwood, only to see it gouged up in days.

I've finished about 700 square feet and I'm down to the last 100-150 square feet, but have been delayed by the uneveness of the concrete subfloor. So, I've been using leveling compound on the floor, and it's been a 2 week hit to my project. I hope to finish it up this weekend.

One thing I discovered when looking into laminate flooring is that they all tend to have formaldehyde, which will emit gas (off-gas) after the floor is installed. One of the reasons I ripped out the carpet is that my son has asthma, so anything making the air quality worse isn't going to work. I discovered DuPont's RealTouch laminate flooring, which is GreenGuard certified as having a minimal impact on indoor air quality. It's held up great so far even with our Australian Shepherd racing around the house with the kids. Not a scratch or gouge.

We also went with GreenGuard certified paint, Dutch Boy Refresh. Again, this means it will emit a minimal amount of noxious gasses while it dries, while at the same time, minimizing noxious gasses in the bathroom ;)

Update 6/24/10
Finally, I was laid off yesterday as expected. As soon as I walked in, I saw facilities and HR people in the building, so I went straight to packing up the last of my stuff. It's good timing since the weather's great, I can spend time with my kids and besides, I've got a floor to finish and walls to paint.

Goodbye AT&T You SUCK

At our 2nd house, we've had AT&T DSL and phone for 4 years. It has steadily gone up in price every year and the service still sucked just as bad as the year before. Three and a half years ago, they raised the price for caller ID to $8/month from an already inflated $6/month. That was practically as much as the phone service itself cost. A little over 2 years ago, they jacked up the cost of DSL by 25%. More increases later, our latest bill for 1.5 Mbps service hit $28/month and with the land line, the total ended up over $43/month.

I gave them a call to see if they had any promotions for existing customers. Nope. Then I asked to cancel the landline and run naked DSL. The customer service rep tried her best to sell a load of bullshit. She said "Oh don't do that. This is what all of us here have - the landline and the DSL." "Well, to do that we'd have to disconnect your DSL and landline and we're not sure if we'd be able to reconnect your DSL to the same circuit." What the f*ck does that mean? Do they have some 5 year old kid pulling and connecting DSL lines for them? She tried to discourage that option. I said "So I'd be better off just canceling all my AT&T services.", to which she replied "No, you're better off leaving everything alone."

I gave AT&T the boot yesterday after we got Comcast 12Mbps service. To replace the landline, I picked up a second Magic Jack. Of course, I'm just trading one evil (AT&T) for another (Comcast), but then paying $35/month to run 1.5Mbps naked DSL would be just stupid - if AT&T would have actually been able to figure out how to connect it in the first place.

First Full Month of Solar

As April ends, I can now assess the results of our first full month of having solar power. Specifically, I was very interested to see if my projections for solar production were even close. If you remember, I wrote this blog entry back in October 2009 as I was trying to determine how big a system to get, what sort of output I could expect and what my savings would be. I created this Google Docs spreadsheet to help with all these calculations.

For the month of April, the spreadsheet predicted for the 3.68kW system we got, production of 513 kWH (this actually comes from the PVWatt tool mentioned in the spreadsheet). It didn't look like we'd get there as the 27th and 28th were overcast and rainy days, but the last 2 days of April turned in the highest production so far. Actual production came in at 538 kWH. Based on this, it looks like the analysis I did in the spreadsheet turned out very accurate and as a result, I'm very pleased with our purchase so far.

Dog Sitters: Don't Shortchange Customers With Automated Homes

We recently went on vacation for a few days and had a dog sitter/walker take care of our dog while we were gone. We had the sitter visit our dog during the day to walk, play with, feed and clean up after him. We chose the duration of the visits and were billed accordingly. To get to our dog, the sitter had to open the side gate, which of course, is monitored and logged by our system. I used that information to gauge when a visit started and when it ended. Our security cameras, which are recorded, could also provide an extra confirmation if needed.

Unfortunately, the results are not good. There was not a single visit where the sitter stayed for the duration we paid.

Visit #1: 18% short
Visit #2: 50% short!
Visit #3: 33% short
Visit #4: 26% short
Visit #5: 23% short
Visit #6: 28% short

Overall, we were billed for 28% more than the actual number of minutes. My wife and I were stunned by how the sitter consistently shortchanged our pooch and not just by a couple minutes. For example, a 30 minute visit was really only 15 minutes and a 1 hour visit was cut short by 20 minutes. All those 5 star reviews on Yelp must have been written by friends or completely clueless people (AKA normal people without automation systems).

Our dog is a very important part of our family, and it bothers us more that he was shortchanged on attention than us being ripped off. Needless to say, rather than confront the sitter, we are going to cut our losses and find a new service (and put up a review on Yelp). This brings up the question, is it really possible to find a sitter who actually stays for the agreed upon durations? You are really at the whim of the sitter's honesty and it seems really easy to cheat. How do you guarantee you get what you pay for?

Overcast Day Has Greater Solar Intensity???

It seems counter-intuitive, yet on a cloudless, sunny day, our panels peaked at a lower electricity output than on a day with overcast. In fact, on the cloudy day, our panels maxed out the inverter, causing it to clip. See the graph below where I combined the charts of 2 different days.



The red line shows the nice production curve on a cloudless day, but the green line shows the output on a cloudy day. The green line maxed out earlier in the day and even later in the day than the red line.

The sunlight hitting earth must be reflecting back up to the sky, but the overcast is causing it to reflect back down again. This results in more solar energy hitting the panels. I would assume that the glass over the panels is like this as well - allowing the sunlight to pass, and some light that reflects off the panels would get reflected back to the panels by the glass.