Month / October 2012

by Aaron

Nest Thermostat API using Node JS and Nest API Update

I’ve been asked by a few people for more details on the API Nest Labs uses for their thermostats, especially regarding setting data (and not just polling).

The API uses mostly JSON formatted data POSTed to their web servers.

Authentication

To authenticate, POST the username and password, encoded as form url-encoded:

POST https://home.nest.com/user/login HTTP/1.1
Host: home.nest.com
Proxy-Connection: keep-alive
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Language: en-us
Content-Length: {Length}
Accept: */*
Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

username={email}&password={password}

Adjust the email and password, and the content length to fit. You may need to remove the Accept-Encoding header value if your client cannot accept gzip or deflated responses.

The server responds with a healthy set of basic information (in JSON format):

{
    "is_superuser": false,
    "is_staff": false,
    "urls": {
        "transport_url": "https://{subdomain}.transport.nest.com:443",
        "rubyapi_url": "https://home.nest.com/",
        "weather_url": "http://www.wunderground.com/auto/nestlabs/geo/current/i?query=",
        "support_url": "https://nest.secure.force.com/support/webapp?"
    },
    "limits": {
        "thermostats_per_structure": 10,
        "structures": 2,
        "thermostats": 10
    },
    "access_token": "GIANT TOKEN STRING==",
    "userid": "1234",
    "expires_in": "Wed, 07-Oct-2012 12:08:00 GMT",
    "email": "user@example.com",
    "user": "user.1234"
}

There are a few things you’ll need from the response:

  • transport_url : this is the address for all of the later request that are made. I’d speculate it’s just a specific server in a server farm (likely with server affinity/session)
  • access_token : this is the key for all later requests and grants access to the API
  • userid/user : a unique user ID
  • expires_in : this is the timestamp for when the access token expires

I’m not sure why the “limits” are being sent back to the client.

You can obtain the service URLs at any time:

POST https://home.nest.com/user/service_urls HTTP/1.1
Host: home.nest.com
Authorization: Basic GIANT TOKEN STRING==
Accept-Encoding: gzip, deflate
Accept: */*
Content-Length: 0
Accept-Language: en-us
Connection: keep-alive
Proxy-Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

Just insert an Authorization header with the access_token value.

The response:

{ "urls": { "transport_url": "https://{subdomain}.transport.nest.com:443", "rubyapi_url": "https://home.nest.com/", "weather_url": "http://www.wunderground.com/auto/nestlabs/geo/current/i?query=", "support_url": "https://nest.secure.force.com/support/webapp?" }, "limits": { "thermostats_per_structure": 10, "structures": 2, "thermostats": 10 } }

Nest labs has a special URL at wunderground.com to access the weather.

One of the first requests you might want to send is to get a complete picture of the system: 

GET https://{subdomain}.transport.nest.com:443/v2/mobile/#USER.ID# HTTP/1.1
Host: {subdomain}.transport.nest.com:443
Authorization: Basic GIANT TOKEN STRING==
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-us
Connection: keep-alive
X-nl-protocol-version: 1
X-nl-user-id: #USERID#
Proxy-Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

You’ll make the request to the transport_url and make sure that the Host, Authorization, and X-nl-user-id header values are set appropriately. The Url now must include though:

  • version
  • mobile
  • and the full user id (like user.1234)

So, it will look something like: /v2/mobile/user.1234

It will respond with the mother-load of all JSON payloads. I’ve trimmed the response for my house as I have three thermostats. But the pattern repeats exactly, so it’s easy to extrapolate how the pattern works: 

{ "metadata": { "SERIALNUM1": { "$version": -1262653277, "$timestamp": 1349697004000, "last_connection": 1349697004683, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM2": { "$version": -1868790132, "$timestamp": 1349696678000, "last_connection": 1349696678701, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM3": { "$version": -1581663504, "$timestamp": 1349696680000, "last_connection": 1349696680647, "last_ip": "LAST.IP.ADDRESS" } }, "track": { "SERIALNUM2": { "$version": 1065037529, "$timestamp": 1349696725390, "online": true, "last_connection": 1349696725390, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM3": { "$version": 981680556, "$timestamp": 1349696726266, "online": true, "last_connection": 1349696726266, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM1": { "$version": 1421919505, "$timestamp": 1349697004728, "online": true, "last_connection": 1349697004728, "last_ip": "LAST.IP.ADDRESS" } }, "user_settings": { "#USERID#": { "$version": 370836640, "$timestamp": 1337481029003, "email_verified": true, "tos_accepted_version": 1319500800000, "receive_marketing_emails": true, "receive_nest_emails": true, "receive_support_emails": true, "max_structures": 2, "max_thermostats": 10, "max_thermostats_per_structure": 10, "tos_minimum_version": 1319500800000, "tos_current_version": 1319500800000, "lang": "en_US" } }, "structure": { "#STRUCTURE-UUID#": { "$version": 1797929878, "$timestamp": 1349689810000, "location": "Mount Horeb, WI", "renovation_date": "2000", "country_code": "US", "away_timestamp": 1349302501, "away": false, "house_type": "family", "name": "Home", "postal_code": "#POSTALCODE#", "creation_time": 1324159145719, "num_thermostats": "3", "devices": ["device.SERIALNUM3", "device.SERIALNUM1", "device.SERIALNUM2"], "user": "user.#USERID#", "away_setter": 1 } }, "link": { "SERIALNUM3": { "$version": 2122853931, "$timestamp": 1327246591000, "structure": "structure.#STRUCTURE-UUID#" }, "SERIALNUM2": { "$version": -1703839727, "$timestamp": 1324159215000, "structure": "structure.#STRUCTURE-UUID#" }, "SERIALNUM1": { "$version": -459415854, "$timestamp": 1325967612000, "structure": "structure.#STRUCTURE-UUID#" } }, "device": { "SERIALNUM1": { "$version": -81037153, "$timestamp": 1349696605000, "heatpump_setback_active": false, "emer_heat_enable": false, "local_ip": "LOCAL.IP.ADDRESS", "switch_system_off": false, "away_temperature_high": 27.778, "temperature_lock_high_temp": 22.222, "cooling_source": "electric", "leaf_threshold_cool": 0.0, "fan_cooling_state": false, "note_codes": [], "heater_source": "gas", "compressor_lockout_leaf": -17.8, "has_x3_heat": false, "target_humidity_enabled": false, "heat_x3_source": "gas", "alt_heat_delivery": "forced-air", "fan_mode": "auto", "has_x2_heat": false, "rssi": 67.0, "emer_heat_delivery": "forced-air", "heatpump_savings": "off", "pin_y2_description": "none", "filter_reminder_enabled": false, "capability_level": 3.0, "schedule_learning_reset": false, "has_x2_cool": false, "hvac_pins": "W1,Y1,C,Rc,G", "ob_orientation": "O", "range_enable": true, "auto_away_enable": true, "dual_fuel_breakpoint_override": "none", "lower_safety_temp_enabled": true, "has_fan": true, "dehumidifier_state": false, "range_mode": false, "nlclient_state": "", "emer_heat_source": "electric", "heatpump_ready": false, "available_locales": "en_US,fr_CA,es_US", "current_version": "3.0.1", "learning_state": "slow", "pin_ob_description": "none", "pin_rh_description": "none", "has_alt_heat": false, "pin_y1_description": "cool", "humidifier_state": false, "backplate_serial_number": "#BACKPLATE-SERIALNUMBER1#", "has_x2_alt_heat": false, "heat_x3_delivery": "forced-air", "leaf_threshold_heat": 19.336, "has_emer_heat": false, "learning_mode": true, "leaf_learning": "ready", "has_aux_heat": false, "aux_heat_source": "electric", "backplate_bsl_info": "BSL", "alt_heat_x2_source": "gas", "pin_c_description": "power", "humidifier_type": "unknown", "pin_w2aux_description": "none", "country_code": "US", "target_humidity": 35.0, "heat_x2_delivery": "forced-air", "lower_safety_temp": 4.444, "cooling_x2_source": "electric", "equipment_type": "gas", "heat_pump_aux_threshold": 10.0, "alt_heat_x2_delivery": "forced-air", "heat_pump_comp_threshold": -31.5, "learning_days_completed_cool": 116, "backplate_bsl_version": "1.1", "current_schedule_mode": "HEAT", "hvac_wires": "Heat,Cool,Fan,Common Wire,Rc", "leaf": false, "type": "TBD", "pin_g_description": "fan", "switch_preconditioning_control": false, "click_sound": "on", "aux_heat_delivery": "forced-air", "away_temperature_low_enabled": true, "heat_pump_comp_threshold_enabled": false, "preconditioning_ready": true, "has_dehumidifier": false, "fan_cooling_enabled": true, "leaf_away_high": 28.88, "fan_cooling_readiness": "ready", "device_locale": "en_US", "temperature_scale": "F", "error_code": "", "preconditioning_active": false, "battery_level": 3.93, "away_temperature_high_enabled": true, "learning_days_completed_heat": 149, "pin_star_description": "none", "upper_safety_temp_enabled": false, "preconditioning_enabled": true, "current_humidity": 45, "dual_fuel_breakpoint": -1.0, "postal_code": "#POSTALCODE#", "backplate_mono_version": "4.0.5", "alt_heat_source": "gas", "aux_lockout_leaf": 10.0, "has_heat_pump": false, "heater_delivery": "forced-air", "auto_away_reset": false, "away_temperature_low": 14.444, "radiant_control_enabled": false, "temperature_lock": false, "upper_safety_temp": 35.0, "time_to_target_training": "ready", "dehumidifier_type": "unknown", "target_time_confidence": 1.0, "temperature_lock_low_temp": 20.0, "pin_w1_description": "heat", "forced_air": true, "temperature_lock_pin_hash": "", "leaf_type": 1, "backplate_mono_info": "TFE (BP_DVT) 4.0.5 (root@bamboo) 2012-09-18 18:18:23", "has_dual_fuel": false, "learning_time": 2113, "creation_time": 1325966794212, "has_humidifier": false, "learning_days_completed_range": 0, "leaf_schedule_delta": 1.11, "user_brightness": "auto", "leaf_away_low": 13.92, "pin_rc_description": "power", "serial_number": "SERIALNUM1", "mac_address": "18b43004f391", "heat_x2_source": "gas", "time_to_target": 0, "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true }, "SERIALNUM3": { "$version": 2134103145, "$timestamp": 1349695665000, /* same as previous */ "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true }, "SERIALNUM2": { "$version": -1340728480, "$timestamp": 1349692217000, /* same as previous */ "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true } }, "schedule": { "SERIALNUM3": { "$version": -1130522241, "$timestamp": 1349692663000, "days": { "3": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "2": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "1": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "0": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "6": { "3": { "time": 67500, "entry_type": "setpoint", "temp": 18.333, "type": "HEAT" }, "2": { "time": 28800, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 24300, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 75600, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "5": { "3": { "time": 67500, "entry_type": "setpoint", "temp": 18.333, "type": "HEAT" }, "2": { "time": 28800, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 24300, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 75600, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "4": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } } }, "schedule_mode": "HEAT", "name": "Basement Current Schedule", "ver": 2 }, "SERIALNUM2": { "$version": -462155699, "$timestamp": 1349674697000, "days": { /* SAME AS ABOVE */ } }, "schedule_mode": "HEAT", "name": "Second Floor Current Schedule", "ver": 2 }, "SERIALNUM1": { "$version": 2014520777, "$timestamp": 1349695806000, "days": { /* SAME AS ABOVE */ } }, "schedule_mode": "HEAT", "name": "First Floor Current Schedule", "ver": 2 } }, "shared": { "SERIALNUM3": { "$version": -493517056, "$timestamp": 1349696367000, "auto_away": 0, "auto_away_learning": "training", "hvac_heat_x3_state": false, "hvac_alt_heat_state": false, "compressor_lockout_enabled": false, "target_temperature_type": "heat", "hvac_heater_state": false, "hvac_emer_heat_state": false, "can_heat": true, "compressor_lockout_timeout": 0, "hvac_cool_x2_state": false, "target_temperature_high": 24.0, "hvac_aux_heater_state": false, "hvac_heat_x2_state": false, "target_temperature_low": 20.0, "target_temperature": 14.444, "hvac_ac_state": false, "hvac_fan_state": false, "target_change_pending": false, "name": "Basement", "current_temperature": 18.11, "hvac_alt_heat_x2_state": false, "can_cool": true }, "SERIALNUM1": { "$version": -1432433268, "$timestamp": 1349696363000 /* SAME AS ABOVE */ }, "SERIALNUM2": { "$version": 2060664119, "$timestamp": 1349696709000 /* SAME AS ABOVE */ } }, "user_alert_dialog": { "#USERID#": { "$version": -1852987123, "$timestamp": 1327246591000, "dialog_data": "", "dialog_id": "confirm-pairing" } }, "user": { "#USERID#": { "$version": 209478897, "$timestamp": 1324159145000, "name": "EMAILADDRESS", "structures": ["structure.#STRUCTURE-UUID#"] } } }

Setting a Temperature

Changing a thermostat’s current set point is easy.

You’ll need the Serial Number (shown as SERIALNUM1 in the JSON above) of the thermostat. 

POST https://275be85a.transport.nest.com:443/v2/put/shared.01AA01AB4611009P HTTP/1.1
Host: 275be85a.transport.nest.com:443
Accept-Language: en-us
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1
X-nl-base-version: 2060664119
Accept: */*
Content-Type: application/json
X-nl-protocol-version: 1
X-nl-user-id: 7236
X-nl-session-id: ios-7236-371385438.528577
Connection: keep-alive
X-nl-merge-payload: true
Authorization: Basic GIANT TOKEN STRING==
Content-Length: 60
Proxy-Connection: keep-alive
Accept-Encoding: gzip, deflate

{"target_change_pending":true,"target_temperature":16.11111}

Set the temperature in Celsius.

There’s also a polling subscription that happens. It’s extremely chatty and from the amount of polling it does, you’d think that the UI was doing live graphing of micro-temperature changes.

Essentially, the polling sends a series of keys, with timestamps, representing the various types of data being requested.

It looks something like this:

{ "keys": [{ "key": "user.#USERID#", "version": 209478897, "timestamp": 1324159145000 }, { "key": "user_settings.#USERID#", "version": 370836640, "timestamp": 1337481029003 }, { "key": "user_alert_dialog.#USERID#", "version": -1852987123, "timestamp": 1327246591000 }, {

It repeats for sections such as “shared”, “message”, “device”, “track”, and more. For my purposes, shared is the winner as it contains the current temperature.

 

Node API

I decided to write a new demonstration application and polish it up a bit, this time using Node.

I’m not going to document the API that I created here (not right now), but here’s a sample of how it can be used: 

var username = process.argv[2];
var password = process.argv[3];

if (username && password) {
    username = trimQuotes(username);
    password = trimQuotes(password);
    nest.login(username, password, function (data) {
        if (!data) {
            console.log('Login failed.');
            process.exit(1);
            return;
        }
        console.log('Logged in.');
        nest.fetchStatus(function (data) {
            for (var deviceId in data.device) {
                if (data.device.hasOwnProperty(deviceId)) {
                    var device = data.shared[deviceId];

                    console.log(util.format("%s [%s], Current temperature = %d F target=%d",
                        device.name, deviceId,
                        nest.ctof(device.current_temperature),
                        nest.ctof(device.target_temperature)));
                }
            }
            subscribe();
        });
    });
}

function subscribe() {
    nest.subscribe(subscribeDone);
}

function subscribeDone(deviceId, data) {
    if (deviceId) {
        console.log('Device: ' + deviceId)
        console.log(JSON.stringify(data));
    }
    setTimeout(subscribe, 2000);
}

The example code runs forever. Smile

I’ve included two methods in the API, “get” and “post” which make it simple to call additional web services that I haven’t yet provided.

Find the code here: https://github.com/wiredprairie/unofficial_nodejs_nest.

Update: there’s a npm as well now (Dec 20, 2012)

by Aaron

Nest Update #12: Software at 3.0 with New Features

As the blogosphere exploded yesterday with news of a second generation Nest thermostat and a new major version of the software (for the thermostats and the controllers such as the web site and various SmartPhones), I wondered what impact the new software and hardware would have on average users, like us.

Do check out the blog post though for full details as there are number of new features in the new device that aren’t available to first generation owners (especially as it relates to supporting a variety of HVAC systems).

image

Before the Nest thermostat was announced and all of the news about lack of support for various HVAC systems, I hadn’t heard of a second-stage cooling, third staging heating, etc. I’d never had them and didn’t know they existed! Smile Now, Nest claims to support up to 95% of HVACs installed in the USA (& Canada?).

I updated my iPhone to 3.0.1 of the Nest App this morning and checked out the new features. Two of my three thermostats had updated to firmware 3.0.1.

IMG_0630

I looked through one of the thermostat’s menus and while there are a few changes (new features), nothing major has changed in the interface. The overall usability is still quite good although I wonder about discoverability of features as the number of features grows.

The home screen hasn’t changed much at all. Still, the giant house:

IMG_0624

The user interface still requires rotation to horizontal to perform anything but the basic changes (such as temperature).

IMG_0625

I don’t know why the Home icon needs to be so prominent as it takes up valuable screen space on a tiny device (and has only a few useful features).

One of the new features is that for a given thermostat, you can actually toggle the fan to ON now if you want. Occasionally we missed that feature (from our old thermostats) when I’ve cooked something that causes an odor to, politely, linger, for a while longer than we’d like. Smile

The feature is buried though under the SETTINGS for a thermostat, and then select AT A GLANCE.

IMG_0626

Toggle it to turn the fan on temporarily.

The remaining values are:

  • Outside temperature (50F)
  • Current inside temperature (68F)
  • Current Humidity (46%)
  • Current set point temperature (69F)

Under the menu, NEST SENSE, you’ll find a one big new thing and a few layout/naming changes:

IMG_0627

EARLY-ON!

IMG_0628

Early-on is a feature that many early adopters had wanted and expected from a modern thermostat. We slapped our heads, cried, complained, hugged…, dismayed that it wasn’t there. And, glory to Nest Labs, they finally added it.

Now, your house can actually be warm/cool when you want. For example, it can be warm when you get out of bed, not just start warming when you get out of bed. This is a HUGE add and I’m very glad Nest has finally added it!

The schedule remains unchanged:

IMG_0629

The overlapping circles still look a bit cluttered to my eye, but it gets the job done.

Under the Home Settings, you’ll find what amount to some survey questions:

 

IMG_0633

IMG_0635

IMG_0637

I’m very pleased with the upgrade to the firmware and smart phone/tablet software. It adds some absolutely needed features.

As an update to my experience with Nest, I’ve definitely had fewer problems lately than I had during the first 6 to 8 months. While I still don’t applaud Nest for their activity in social media and reaching out to their customers proactively (as they never responded to any of the comments here), they have been active behind the scenes.

The wireless connectivity to the thermostats has improved. I haven’t noticed the same problems as before and I THANKFULLY haven’t had to re-add my account or Wifi information in months to any of our thermostats.

I’ve had enough good success recently to change my recommendation on Nest Thermostats. If you’re in the market for a new thermostat, and you’ve got $249US to spend, I’d say it should definitely be a strong contender.

Read through the comments and the issues – but understand a lot of the issues have been resolved.

You can buy the older model while supplies last for $229.

The second generation is available for preorder from Amazon today.

I think it’s ready to be part of your house (after you check your system’s compatibility).

If you have found these posts useful, please consider using the Amazon links above to buy your shiny new Nest thermostat (especially if you have Prime!) as a way of saying thanks!

What do you think of the updates and new hardware?