How to debug an underscore.js template

Given a simple template like this:

<div class="solution" data-question-id="<%= model.get('Id') %>">    
    <div class="title"><%= model.get('Name') %></div>
    <div class="company"><%= model.get('Company') %></div>
    <div class="version"><%= model.get('Version') %></div>
    <div class="detail"><%= model.get('Summary') %></div>
    <div class="actions">
        <a href="/#solution/<%= Id %>/<%= model.get('Id') %>">Detail</a>
    </div>
</div>

And using your favorite JavaScript interactive debugger (Visual Studio 2012 is my favorite when I’m doing MVC 4 Razor development), just add a debugger statement within your template temporarily:

<div class="solution" data-question-id="<%= model.get('Id') %>">
    <% debugger; %>
    <div class="title"><%= model.get('Name') %></div>

Assuming debugging is enabled, this will break (in Visual Studio for example) on the debugger line whenever your code template is executed.

The emitted template code thankfully has line feeds embedded so it’s readable:

function anonymous(obj,_) {
var __t,__p='',__j=Array.prototype.join,print=function(){__p+=__j.call(arguments,'');};
with(obj||{}){
__p+='<div class="solution" data-question-id="'+
((__t=( model.get('Id') ))==null?'':__t)+
'">\r\n    ';
 debugger; 
__p+='\r\n    <div class="title">'+
((__t=( model.get('Name') ))==null?'':__t)+
'</div>\r\n    <div class="company">'+
((__t=( model.get('Company') ))==null?'':__t)+
'</div>\r\n    <div class="version">'+
((__t=( model.get('Version') ))==null?'':__t)+
'</div>\r\n    <div class="detail">'+
((__t=( model.get('Summary') ))==null?'':__t)+
'</div>\r\n    <div class="actions">\r\n        <a href="/#solution/'+
((__t=( Id ))==null?'':__t)+
'/'+
((__t=( model.get('Id') ))==null?'':__t)+
'">Detail</a>\r\n    </div>\r\n</div>\r\n';
}
return __p;

}

You’ll see the debugger emitted in-line. It’s very handy for inspect the values of variables, objects, etc. In the example above, I wanted to confirm that the model being passed was in the proper format.

I’ve used it many times to help debug a template that wasn’t working the way I’d expected.

This also works well in Chrome’s debugger.

Smarter queuing of files using jQuery deferred and when

I had a somewhat simple need …a single web page style application using Backbone.JS had multiple template files it needed to download, if and only if the files hadn’t already been downloaded before.

While I suppose I could have relied on browser caching, I wanted to manage the requests in JavaScript. The trick was that at any given time, a request might include one or more template files, and some of the request might have already been made, or in progress.

Here’s the queuing function:

(function() {
    var allFiles = {};
    var allResults = {};

    var queueRequest = function (path, files, options) {
        options = options || {};
        files = $.isArray(files) ? files : [files];
        
        // this will contain all of the deferreds that will be 
        // wrapped with a `when` deferred
        var when = [];
        // might want to override some defaults...
        var ajaxOpt = $.extend({
            dataType: 'json',
            type: 'GET'
        }, options.ajax || {});

        // go thru each file
        $.each(files, function (i, file) {
            var xhr;
            // if we've not seen it before
            if (typeof allFiles[file] === 'undefined') {
                // kick it off
                xhr = $.ajax(path + file, ajaxOpt).done(function (data) {
                    // store the file results in the hash index
                    allResults[file] = data;
                });
                // keep the deferred for later use
                allFiles[file] = xhr;
                // and keey this for later
                when.push(xhr);
            } else {
                // already seen this, so just pack it on
                when.push(allFiles[file]);
            }
        });
        // return all of the built up deferreds as a 
        // single when. this will then wait until 
        // all are satisfied
        return $.when.apply(this, when).done(function () {
            if ($.isFunction(options.done)) {
                options.done.call(this, allResults);
            }
        }).fail(options.error);
    };

    window.Queue = queueRequest;
})();

Queuing is easy enough:

Queue(app_url("api/template/"), templates, {
    done: function (allFiles) {
        _.each(templates, function (n) {
            // compile only if needed
            if (typeof allTemplates[n] === 'undefined') {
                allTemplates[n] = _.template(allFiles[n]);
            }
        });
        if (typeof callback === 'function') {
            callback.call(context, allTemplates);
        }
    }
});

allTemplates is a list of compiled underscore templates stored elsewhere.

The first parameter to the Queue function is the full path to the templates. I’ve got a simple MVC controller that responds with a template given a key, so that’s the path that is provided in the example above. The list of templates (as an array) is passed.

The Queue code returns a jQuery deferred when. The function returns both active and completed ajax requests … which means that the jQuery when function may immediately fire … or not. When simply waits for all of the async operations to complete, and then calls the callback passed to done.

When complete, the code passes all results back to the requesting function. It’s not filtered as I trusted the caller (no reason to unnecessarily clone the array and waste CPU time). :) 

How to rewrite a MongoDB C# LINQ with a Projection Requirement using a MongoCursor

The LINQ Provider for MongoDB does not currently take into account data projections efficiently when returning data. This could mean that you’re unnecessarily returning more data from the database than is needed.

So, I’m going to show you the pattern I applied as a replacement for the LINQ queries when I need to use a projection.

Given the following simple LINQ statement:

var query = 
    (from r in DataLayer.Database
         .GetCollection<Research>()
         .AsQueryable<Research>()
        where !r.Deleted
        select new
        {
            Id = r.Id,
            Title = r.Title,
            Created = r.Created
        }).Skip(PageSize * page).Take(PageSize);

it can be converted to a MongoCursor style search like this:

var cursor = DataLayer.Database.GetCollection<Research>()
    .FindAs<Research>(Query<Research>.NE(r => r.Deleted, true))
        .SetFields(
            Fields<Research>.Include(
                r => r.Id, 
                r => r.Title, 
                r => r.Created))
        .SetLimit(PageSize)
        .SetSkip(PageSize * page);           

I’ve attempted to format it in a similar way mostly for my own sanity. As you see, both queries first get access to the database collection. But, instead of using AsQueryable<T>, you’ll use the FindAs<T> method. The query is more verbose in the second example, although not overly so. I chose to keep it strongly typed by using the generic version Query<Research>. By doing so, it meant that I could use an Expression to set the field/property that was being queried, rather than rely on a string (I could have just passed “Deleted” as a string in the code).

By strongly typing the parameters in this way, it meant that the compile process can catch things like type mismatches (verifying that the value being compared is a Boolean for example as is the Deleted property).

Secondly, and this addresses the requirement of a result projection, I’ve included just those fields that are required by the UI rather than all of the fields in the document. One of the fields is potentially quite long (up to 1MB of text), and in this case, unnecessary in a summary list display in my web application. Here, I used the SetFields method of the MongoCursor.

The C# driver includes a static class called Fields (in a generic and non-generic form) which can be used to express the set of fields to be included/excluded. I’ll point out that there is an option to just pass in a list of strings to SetFields, but it’s not strongly typed. So again, no compile-time checking that I’ve got the property names correct. I’m going for safety here, so I chose the strongly-typed generic implementation of Fields<Research>. Then, using the Expression syntax again, I’ve selected the fields I needed as a parameter list.

Finally, I added some code to limit the result size and set the skip equivalent to the original LINQ query.

There are a number of other Query methods that you can use to build more complex operations.

For example:

var q = Query.And(
    Query<Research>.Exists(r => r.Title), 
    Query<Research>.Matches(
        r => r.Title, BsonRegularExpression.Create(new Regex("^R"))));

The above maps to the following MongoDB query:

{ "$and" : [{ "Title" : { "$exists" : true } }, { "Title" : /^R/ }] }

Title field exists and the Title field starts with an uppercase “R”.

While the Query style syntax is more verbose than the equivalent LINQ statement, the result still is expressive and very readable and maintainable.

FYI: If there’s an index on Title, then the /^R/ syntax returns the results the most efficiently in MongoDB (as it stops searching after the first character).

How to view the MongoDB Query when using the C# LINQ Provider

If you’re using the Official MongoDB C# Driver from 10gen, you may want to occasionally verify that the generated query matches your LINQ query (or at least that it’s building something efficient).

Take for example this query:

var query = 
    (from r in DataLayer.Database.GetCollection<Research>().AsQueryable<Research>()
        where !r.Deleted
        select new
        {
            Id = r.Id,
            Title = r.Title,
            Created = r.Created
        }).Skip(PageSize * page).Take(PageSize);

query.DebugWriteMongoQueryText();

I wanted to verify that the query was checking the Deleted property, and see if the projection was considered as part of the query, so a few lines of code later …

public static string ToMongoQueryText<TQueryable>(this IQueryable<TQueryable> query)
{
    return (query as MongoQueryable<TQueryable>).GetMongoQuery().ToString();            
}

[Conditional("DEBUG")]
public static void DebugWriteMongoQueryText<TQuerable>(this IQueryable<TQuerable> query)
{
    Debug.WriteLine("QUERY: " + query.ToMongoQueryText());
}

I added two extension methods to IQueryable. As the return type for the query in my sample is actually an anonymous type, it wasn’t something I could easily type in the Immediate Window in Visual Studio for example.

<>f__AnonymousType2<string,string,System.DateTime>

No thanks  <>f__AnonymousType2<string,string,System.DateTime>.

I added the Conditional Attribute so that the code would only execute in a Debug build of my application.

query.DebugWriteMongoQueryText();

The output of the query above was:

QUERY: { "Deleted" : { "$ne" : true } }

As you’ll see, the anonymous projection wasn’t considered. Unfortunately, that’s a known issue with the current driver. This mattered in my above example, as some of the fields could contain a large amount of data that wasn’t needed by the current display.

Knockout binding for JavaScript route fixup

Part one.

After the first round, I felt compelled to KnockOut the code a bit more. I’d mentioned I wasn’t pleased with the code exactly. It needed some refactoring.

So, I’ve created a new Knockout binding handler. This binding handler replaces  named parameters with a model’s properties in a path.

For example, given this object:

Property Name Value
id A123
first_name Aaron
state WI

The following paths would be converted thusly:

Original Replaced
/person/{id} /person/A123
/person/{state}/{id} /person/WI/A123

You get the idea. Here’s the JavaScript code:

ko.bindingHandlers['route'] = {
    // Examples: 
    //      <a data-bind="route: {model: $data, url: 'person_details', attr: 'href' }" >
    // or, you can shortcut the syntax to default to the currently bound object and just pass 
    // the url or the route name as a string directly
    //      <a data-bind="route: 'person_details' }" >
    update: function (element, valueAccessor, allBindingsAccessor, $data) {
        var valueUnwrapped = ko.utils.unwrapObservable(valueAccessor());
        var options = ko.bindingHandlers.route.options || {};

        // look for the model property 
        var model = ko.utils.unwrapObservable(valueUnwrapped['model']);
        if (typeof model !== 'object') {

            model = ko.utils.unwrapObservable($data);
            if (typeof model === 'undefined' || model === null) {
                throw new Error('set route model to object (or nothing bound?)');
            }
        }

        // look for the url property first
        var url = ko.utils.unwrapObservable(valueUnwrapped['url']);
        // validate we've got something as a url (might be a name, might be a full url)        
        if (typeof url !== 'string' || url == "") {
            url = valueUnwrapped;
            if (typeof url !== 'string' || url == "") {
                throw new Error("set route url property to route name or url directly");
            }
        }

        // is it on the keyed collection?
        var map = options.map;
        if (typeof map !== 'undefined' && map !== null) {
            if (map.hasOwnProperty(url)) {
                url = map[url];
            }
        }
        // check for a routing function as well
        var fn = options.routeNameToUrl;
        if (typeof fn === 'function') { url = fn.call(null, url); }
        // did we get something meaningful?
        if (url !== null && url !== '' && url.length > 0) {
            url = ko.bindingHandlers.route.buildUrl(url, model);            
        }
        // the url might need some fixin after a routing, anything goes here (might just be a default)
        fn = options.fixUrl;
        if (typeof fn === 'function') { url = fn.call(null, url); }
        element.setAttribute(ko.utils.unwrapObservable(valueUnwrapped['attr']) || 'href', url);
    },    

    // given a model, this function replaces named parameters in a simple string 
    // with values from the model
    //     /path/to/some/{id}/{category}
    // with object { 'id' : 'abc', 'category' : 'cars' }
    // becomes
    //     /path/to/some/abc/cars
    buildUrl : function(url, model) {
        // unfixed if there's not a thing
        if (typeof model === 'undefined' || model === null) { return url; }

        var propValue;
        for (var propName in model) {                
            if (model.hasOwnProperty(propName)) {
                propValue = model[propName];
                if (ko) { propValue = ko.utils.unwrapObservable(propValue); }

                if (typeof propValue === 'undefined' || propValue === null) {
                    propValue = "";
                } else {
                    propValue = propValue.toString();
                }
                url = url.replace('{' + propName.toLowerCase() + '}', propValue);
            }
        }
        return url;
    },

    options: {
        // ** convert a route name to a url through whatever means you'd like
        // routeNameToUrl : function(routeName) { return url; } 

        // ** anything you want, called after routeNameToUrl, might add a virtual directory
        // ** for example
        // fixUrl: function(url) { return url;  }  

        // ** A map route names to URLs **
        // all other functions are called if set (to possibly override this)
        // this is not required if you use one of the other functions
        // map : { 'a_route_name' : '/path/to/something/{id}/{action}' }        
    }
};

In another JavaScript file, I did initialize some of the options:

ko.bindingHandlers.route.options.routeNameToUrl = getRoute;
ko.bindingHandlers.route.options.fixUrl = app_url;

The getRoute function just maps a route name to a path, and the app_url prepends the virtual directory to the path as needed.

Here it is in use:

<div data-bind="foreach: data.persons">
    <h3 class="title" data-bind="text: Title"></h3>
    <div>
        <a data-bind="route: { model: $data, url: 'person_details', attr: 'href' } ">Details2</a>
        <a data-bind="route: '/data/details/{id}/{title}' ">Details</a>
        <a data-bind="route: 'person_details' ">Details</a>
    </div>
</div>

You’ll probably like the new way better syntactically at least compared to the old way. A route binding requires one input when used in it’s most basic form:

  • url = the URL or route name to use as the template for the replacement. It should contain (or later resolve to) curly-braced enclosed property name keys which will be substituted by values from the model. The value of the property could be either a route name (see options below) or a path.

When using just the basic form you can use the shortened syntax:

<a data-bind="route: 'research_details' ">Details</a>

This handily binds to the current object (via the bindingContext.$data property of the bindingHandler update function call, which is also the fourth parameter, which I’ve renamed to $data rather than the typical viewModel). So, you won’t need to necessarily (explicitly) set the model for the binding.

If you need a a bit more control, you can change the syntax and get access to a few other options (including direct access to the model you want to bind to if the current item isn’t directly what you want to use).

  • model = this is the object that contains the properties and values to be used as replacements within the url
  • attr (optional) = the name of the attribute to set the generated url into. Defaults to href if not set.

There are a few global options you can control as well:

  • routeNameToUrl = (function)(routeName) optionally, given a route name, should return the path (or the original value). Here you can do a lookup of routeName to path.
  • map = {object} the object properties should be route names and set equal to the path. this optional lookup is performed before the routeNameToUrl function is called.
  • fixUrl = (function)(url) do anything here. this is called after the mapping and routeNameToUrl is optionally called. I use this to correct javascript Ajax request paths by appending the application virtual directory

(Thanks to Ryan for the suggestion to use the $data on the bindingContext, and then again for the nudge to just use the 4th parameter. Smile )