/#!

Lightweight Relation Modeling with Backbone

I’ve been using Backbone.js more and more in my day-to-day development for a myriad of reasons. It’s a wonderful balance of a light footprint and a set of powerful, flexible tools. One of the most frustrating things for me is when a framework prevents me from doing something… I don’t know that I’ve had a case of that with Backbone so far. However, because Backbone is so trim and so new, developers have to solve a lot of novel challenges before being able to fully leverage the framework.

Problems with Complex (Relational) Models

Once you start working more with Backbone models as client-side mirrors of your persistence layer (if only structurally), you run in to the problem of what to do with relationships between model types.

Take, for example, this JSON object representing an album.

var album_json = {
	name: "My Album",
	releaseYear: "2010",
	trackList: [
		{
			name: "My First Track",
			order: 0,
			duration: 3600,
			streamUrl: "http://example.com/stream.mp3"
		}
	]
};

The album has some metadata (name, releaseYear), but also a list of tracks. The tracks themselves look like models. The trackList looks like a collection.

How would we model this with Backbone? Well, we’d probably start with something obvious, like this:

var Track = Backbone.Model.extend(),
	TrackList = Backbone.Collection.extend({
		model: Track
	}),
	Album = Backbone.Model.extend();

Then things get a little tricky. So the Album looks like it needs to have some internal attributes for `name` and `releaseYear`, but then it needs a reference to a TrackList for the `trackList` property. How do we set that up? More importantly, how do we ensure that any future operations on the model (syncing from the backend, manually setting properties on the model, etc) get delegated correctly to the TrackList, instead of overriding it with dumb JSON values? If you don’t see what I mean, think about what happens here:

var model = new Album(album_json);
model.set({
	trackList: [{
		name: "A different track",
		order: 0,
		duration: 1800,
		streamUrl: "http://example.com/stream2.mp3"
	}]
});

Not only is it messy, but unless we’re doing some magic, our nice TrackList is going to get completely wiped out by this call, and we’re left with this plain ol’ JSON array.

Existing solutions

I’ve been aware of Backbone-relational for some time now, and in doing some research for this post I came across another called Ligament. Both of these implementations claim to support pretty much any relational whim you would have. Ligament seems way less mature than Backbone-relational; I’m contemplating leaving it off this list because it really only supports reads, and relies on everything being set up in a precise manner before any operations can start happening.

Backbone-relational is very powerful, but also very esoteric and has a few design decisions that I disagree with:

• Bi-directional support.
  Both implementations work to bring full bi-directional support to models, so that if I loaded, say, a track from the server, and it had a reference to its parent through some special id key:

  {
      "album_id": "1",
      "name": "My Track",
      "duration": 3600,
      "streamUrl": "http://example.com/stream.mp3"
  }
  

  The library is ‘smart’ enough to figure out how to fetch that album just off of the album_id. Which brings me to my next point.

• Too much magic.
  You only have to look at the GitHub issues page for Backbone-relational to understand the problem with this. Seems like the only person who really knows how this thing is working is the original developer. The problem is that you really have to understand Backbone’s own internal control flow (when events are fired, what happens in the constructor, etc) and then figure out how Backbone-relational is augmenting that with its own magic. The end result is you have weird cases where models are automagically fetched, and their events are being suppressed by stuffing everything into a blocking event queue where locks are acquired at the beginning of every major model operation, because otherwise your handlers would execute with the wrong data or your collection would scream because it was trying to add two of the same model (which makes Backbone throw a nasty error).
• All or nothing.
  Wouldn’t be so bad if the implementation wasn’t so cryptic, but in order to use these libraries you have to make everything inherit from these new model prototypes. The idea of having to wade through this extra layer of code on debugging is not enticing.

K.I.S.S.

Here’s an alternate implementation for this relational problem that reduces the code you need to write, while keeping it dirt-simple to understand what’s going on.

function delegateModelEvents(from, to, eventKey) {
	from.bind('all', function(eventName) {
		var args = _.toArray(arguments);
		if (eventKey) {
			args[0] = eventKey + ':' + args[0];
		}
		to.trigger.apply(to, args);
	});
}

function getUpdateOp(model) {
	return (model instanceof Backbone.Collection) ? 'reset' : 'set';
}

Backbone.RelationalModel = Backbone.Model.extend({
	relations: {},
	set: function(attrs, options) {
		_.each(this.relations, function(constructor, key) {
			var relation = this[key];

			// set up relational model if it's not there yet
			if ( !relation) {
				relation = this[key] = new constructor();

				// makes it so relation events are triggered out
				// e.g. 'add' on a relation called 'collection' would
				// trigger event 'collection:add' on this model
				delegateModelEvents(relation, this, key);
			}

			// check to see if incoming set will affect relation
			if (attrs[key]) {
				// perform update on relation model
				relation[ getUpdateOp(relation) ](attrs[key], options);

				// remove from attr hash, prevents duplication of data +
				// keeps models out of attributes, which should be only used for
				// dumb JSON attributes
				delete attrs[key];
			}
		}, this);

		return Backbone.Model.prototype.set.call(this, attrs, options);
	}
});

The reason this works is because `set` is used internally by Backbone for any operation that updates a model. That means the constructor, where the attributes are set up initially, any Backbone.sync responses that originate from a fetch/save call, and of course just calling `set` directly. So we have overridden one method to just be a little smarter, and immediately there are huge gains for this problem. Going back to my previous example, my models would now be this:

var Track = Backbone.Model.extend(),
	TrackList = Backbone.Collection.extend({
		model: Track
	}),
	Album = Backbone.RelationalModel.extend({
		relations: {
			trackList: TrackList
		}
	});

I also added a quick event delegation routine so that, if you wanted to, you could bind on any relation’s events from the top-level model. In my case, I could listen for when any track in an album changed its name:

album.bind('trackList:change:name', function(track) { ... });

My little function is just a quick exercise. The best part about working in Backbone is that you can drastically augment its behaviors just by mixing in a little extra special sauce. It would be trivial to build a more complex event propagation system where all the callbacks for relational events had a reference to the top-level model passed in as one of the arguments, for example.

Caveats

Yes, this solution only works for top-down HasOne/HasMany relations.

I deliberately ignored the problem of bi-directional support, because I don’t see it as a worthwhile problem to solve within the bounds of a one-size-fits-all solution. The level of magic incurred is just too high, and leads to too much instability and confusion for something that should be an edge case. However, I recognize that the problem is still there for some uses – it just so happens that I haven’t had any need for it in any of the work I’ve had to do with APIs. I will probably devote some more time to this problem to come up with a middle-ground solution. It won’t be as ‘powerful’ as Backbone-relational… but that’s the point.

UI Corner: Twitter List Management

When Twitter launched their new web interface (NewTwitter), I finally started using their web client pretty regularly. The new interface is sleek, and really streamlines the timeline in to a much better discovery tool. I can find conversation ‘threads,’ can browse media content in-stream, jump along tangental lines of connections via suggested users and related tweets, yada yada yada. In my opinion, it’s much better.

However, one seemingly integral part of Twitter’s new UI – Twitter Lists – has been swept up in to the corner a bit, and I’m not sure why. As I start to use Twitter more for news, inspiration, and a window to what’s going on in the world, I find that keeping lists of clusters of content a very solid feature. I can have one list for design articles, one for news, one for development blogs… you get the idea. However, the simple act of creating and updating a list is something that will probably frustrate users. Let me first illustrate the problems, and then I’ll propose what I think would be a simple-ish solution that wouldn’t move the essential UI around much.

The Scenario

OK. I’m a user, and I’ve decided I want to make a List just for my co-workers. I already have a list defined in my mind of all my co-workers, I kind of remember which ones have Twitter accounts and which ones don’t, and I have no idea what their usernames are specifically. But I figure I’ll handle that later. First, I just need to find out how to add a list.

To highlight where what seems like the List options are, I searched for the term “List,” which found a few options:

That’s great, and about what I would expect. Clicking the disclosure triangle in the main nav gives me a nice drop-down menu, with a few options.

Keeping in mind that I’m just trying to “add a new list with co-workers,” the option “Create new List” seems applicable. Great! I’ll click it.

Now, Twitter makes the (correct) assumption that I’m wanting to complete a special task regarding making this List, so it focuses my attention on a modal dialog. It’s laid out nicely and the options are very clear to me. I can’t really add people to the List right off the bat, but I’m assuming that will follow shortly — this is just some prep-work…

After I click “Save,” I arrive at what looks like my new List’s “profile” or detail page. At this point, all I want to do is start on the “meat” of this task: adding my co-workers. I see there’s a search box inside where statuses or user account listings would be. I’ll start there, and add my first co-worker. I start typing in the name, and even wait a bit, expecting it to populate with some auto-suggestions based on who I’m currently following, but no dice! That’s kind of a bummer; I was hoping to get everything done from here in one fell-swoop. Whatever, I’ll search for the first guy (Gabe) anyways. I’ll even use his proper (full) name to help the search out…

Well, that’s interesting. Not only was I taken somewhere else entirely (Who to Follow? Shouldn’t it be “Search Results” or something?), but Gabe isn’t even on there! There are quite a few Gabriel Hernandez’s (what is the plural of that?) in the world… it could take ages to go through this list. The good thing is, I actually know Gabe’s username, so I’ll just enter that.

This time, WTF comes through. I see Gabe’s profile, and notice a button on the right of the “Follow” indicator, and it kind of looks like a list, or options, or something, so I’ll check it out. It doesn’t really scream “This is how you add me to a list” but it’s close enough for me to investigate. And I get a nice drop-down again with what looks like a breakdown of my Lists. I see there is a checkbox next to the co-workers list I just made, so I’ll check it. The checkbox is activated, so I’m assuming that Gabe was added to the list. I feel a little unsure just because there was no confirmation. But, it’s easy to check: my co-workers List should have been updated. I’ll go over and look at it again to see if that’s the case.

Cool, well it looks like Gabe is there now! …But where did the search form go? Granted, I didn’t really like it, but it was kind of useful to have it in-place in the List view. I’m on my own to find the rest of the people I want to add.

However I find the target, I have to go through this for each and ever user that I want to add to the list. For the small company I work at, that’s not too bad – only 6 or 7 of my co-workers have Twitter accounts that they like to update. If I were making a large list from scratch it would be a heavier undertaking. Overall, I’m left with the impression that this is just taking too long.

What went wrong?

There were a few things that contributed to what I felt was, at times, a bit of a clunky and time-intensive experience for what I was hoping would be a breeze.

1. Search assistance was a missed opportunity. Conventions like auto-complete and search-as-you-type are being used, to great effect in most high-traffic web services. Mobile apps are also pushing this trend forward (appropriately so; It’s much harder to type on small keypads). I think that implementing a profile-lookup search that brings up user accounts as you type would single-handedly turn this around, and I present some ideas for that during the last part of this analysis.
2. Search pages were not relevant. Search results were not weighted in any way that seemed useful. I expected Twitter users I was following to appear at the top of the list for easy access.
3. Adding additional users was not an obvious process. After at least one user is added to the List, the List page displays a subset of the List’s total tweets, instead of the search form. The result is that it’s not obvious how to add a user to a List from the List UI.

Solutions

Introduce auto-complete to user search

The List view’s search box has one primary purpose: to find other users that I want to add to my List. Moreover, if I already know which users I’m going to add, it doesn’t make sense to make me find each one of them through a search result page and then add from there. As I was typing Gabe’s name, for example, I should have seen something like the following:

Improve search results with relevant content

Even if autocompleted search results are on, it’s a good idea to have a fall-back search that functions similarly in case the autocomplete is broken, the user’s internet connection is slow, etc. Just having users you follow (and maybe users who follow you) at the top would help. Users that you have dm’d in the past is also a candidate here (since you can DM people as long as they follow you, but this is a better metric of interaction that just the fact that a user follows you). By putting those results at the top for this type of search, in which I contend the intent is to search amongst your Twitter contacts, the process becomes more stream-lined.

Keep the improved search at the top of the List view

To distance this new search that assumes an intent of wanting to surface ‘close’ Twitter contacts from the global search, it would help to keep this List-optimized search field in the List UI. This way the search field is tied contextually to the List, and it will be the go-to for this kind of operation. Currently a search box is only visible in the List UI if there are no users added to the List. I think the search feature is valuable enough to warrant placement whenever you’re viewing the List UI page (after all, the edit/delete operations are there anyways… it’s kind of already a content management view).

Why not do this all in one step?

There was a really good opportunity for a rewarding experience in this scenario: the modal dialog that served as the List content primer. Instead of just being able to name the List and provide a description, doesn’t it make sense to also offer a way to pump the list full of people straight away? Generalizing the autocomplete idea, Twitter can have a way to quickly add users from within the modal dialog. Users can quickly add people to the List, remove them if they change their mind, and accomplish everything in one shot. This will also avoid the somewhat awkward ‘blank’ List view that you get when no users have been added yet.

Closing Thoughts

As I’ve worked on this write-up, I’ve found a few different flows and hook-points for accomplishing this task. However, none of the flows address the particular case that I outline in the beginning of this post. Rather, Twitter makes an assumption that the primary user need for Lists is the ability to easily update / maintain them on a user-by-user basis. This is great when somebody you’re following starts to become less interesting to you, and you want to take them off all Lists straight from wherever you’re viewing the content. All you need to do is click through to the profile. This is also good when you find somebody new to follow. You can follow them and add them to a List without having to go to some special corner of the system. I’m not convinced, however, that this use-case is the most common one.

I should also note that the task I’m addressing can be accomplished without too much hassle by going to your Following section and then adding whatever users you want to a new List. This breaks down somewhat if you’re following a lot of people because you have to wade through a lot of noise, and therefore I maintain this use case is hindered by the current offering.

Fortunately, it’s not a giant leap to improve this scenario, because Twitter has done a very excellent job with anticipating user needs and streamlining both content discovery and curation (no small feat). This is just one corner of the UI that could use a spruce-up.

Sealing / Freezing Objects with Javascript 1.8.5

After spending some time poking around the ECMA-262 specification, I realized there was a mountain of functionality coming with ECMAScript-5 that I was completely unaware of. Further research shows I am pretty behind the curve, as John Resig posted a great breakdown of all of the new Object APIs in a blog post almost a year ago.

I recommend reading Mr. Resig’s post for a much better breakdown. To save repetition, I’ll just illustrate some quick example on patterns these new features afford.

Member constants

// quick and dirty...
var Constants = {
	SOME_VALUE: "foo",
	OTHER_VALUE: "bar"
};
Object.freeze(Constants);

console.log(Constants.SOME_VALUE = "changed."); // 'changed.'
console.log(Constants.SOME_VALUE); // 'foo'

console.log(Object.isFrozen(Constants)); // true

// To allow adding constants in the future,
// don't use freeze. Use property descriptors instead.

var Constants_Dynamic = {};
Object.defineProperties( Constants_Dynamic, {
	"SOME_VALUE": {
		value: "foo",
		writeable: false
	},
	"OTHER_VALUE": {
		value: "bar",
		writeable: false
	}
});

console.log(Constants_Dynamic.SOME_VALUE = "changed."); // 'changed.'
console.log(Constants_Dynamic.SOME_VALUE); // 'foo'

Object.defineProperty( Constants_Dynamic, "NEW_CONST", {
	value: "baz",
	writeable: false
});

console.log(Constants_Dynamic.NEW_CONST); // baz

The above example will actually throw errors when you try to assign to non-writeable properties when you are in strict mode (which is a good thing). I would caution against freezing an object unless you absolutely know that you will not need to add properties in the future. There is no way to un-freeze an Object (otherwise it would be quite pointless!). If you’re exposing a library, odds are you don’t absolutely know anything, but it’s still nice to have some read-only protection.

I also found it interesting that the assignments will return the value you tried to set it to, even though the assignment didn’t go through. A little confusing, to be sure.

I’ll be brainstorming some less-contrived examples in the future, but it will be interesting to see how these new constructs will modify the landscape of client applications.

Simple JS Multiton/Factory

The Problem
I wanted to create one interface with different implementation details across several product prototypes. I would only be using one product at any given time, but depending on certain environment variables (detected on page load or some other configuration step) would choose which product to use.

The Solution
Create a simple instance-managing system that had one function – instance(), which returned the default operating product instance. It is also possible to manually request other instances if you know their identifier. It borrows some ideas from Zend-style autoloading for the instance creation step. In order to add new products, you simply add them to the _products dictionary object.

var Multiton = (function() {
	var _instances = {},
		_products,
		_default;

	function create_instance(name) {
		var prefix = 'Product_',
			product_name;

		product_name = prefix + name.slice(0,1).toUpperCase() + name.substring(1);
		if (typeof _products[product_name] === 'function') {
			return new _products[product_name]();
		}

		// warn user of invalid instantiation attempt
		return null;
	}

	_products = {
		Product_Concrete: function Product_Concrete() {
			// product code
		}
	};
	_default = 'concrete';

	return {
		instance: function(name) {
			name = typeof name === 'string' || _default;

			if (typeof _instances[name] === 'undefined') {
				_instances[name] = create_instance(name);
			}

			return _instances[name];
		}
	};
})();

Turning this in to a factory pattern instead of a multiton is trivia; just remove the _instances collection and instead chug out new product objects for every request of some method factory().