What is split testing? The short answer is that it is a comparison test of different versions of some content (on the web, typically a page). The different versions are compared by first showing them at random to users, then measuring success by some means, for example seeing how many lead to a purchase.

There is a split testing module for Drupal, but I think its scope is far too limited to be useful, as all it does is compare two different themes. If you are trying to test variants of some specific content - for example a call to action - rather than the way it is presented, then this module won’t help. (If you do want to measure one theme against another, of course, then it’s ideal.)

I have a partially developed solution of my own to the problem of testing variants on an individual content node, though there are some issues left to solve. One is to find a good answer to the question of caching, so the site as a whole is cached while the pages under test are either not cached at all, or cached in a way that permits different variants.

If anyone is interested in sponsoring further development, or taking over what I have written so far, please let me know. I’d also be interested in discussing other possible approaches, perhaps by using other tools such as Google Website Optimizer (mine is very Drupal specific - it works by altering the content of the node at load time).

New services for measuring the performance of websites seem to appear every week. One reason is the much better support for Javascript in modern browsers, which makes it possible to measure user activity non-intrusively yet comprehensively, simply by adding a little script code to each page.

What about measuring usability, though? Is usability as a concept well-defined enough to be measured? A recent discussion on the Cre8asite forum throws some light on the issue, though in my opinion the two sides here are talking about different things. Usability cannot simply be measured in terms of conversions, because many sites don’t have conversion as a goal.

For example, it is meaningful to ask whether the BBC’s website is more usable than the (appalling, in my opinion) one belonging to Channel 5, and it is reasonable to give a qualitative answer based on experience. That answer could be backed up by quantitative research which would produce real numbers, but as is well known, statistics can be used to prove practically anything. For sites that don’t have narrow goals, asking the right questions about usability is to some extent an art.

An innovative approach to measuring usability is offered by CrazyEgg, which helps you to visualize what site visitors are actually doing. You can see which links and buttons actually get followed, and how users proceed through your site. A lot of this information has been available from other tools including Google Analytics, but CrazyEgg’s views of the data are for the moment at least somewhat smarter.

With tools like this appearing and Google’s Website Optimizer now available to all Adwords users it looks like 2007 may prove to be the year in which website measurement grew up. It is even possible, although I am somewhat sceptical about this, that the increasing use of such tools might finally persuade designers and site owners to avoid user-unfriendly practices like - as on the home page of the Channel 5 site at time of writing - embedded video you can’t turn off.

Google Website Optimizer - a Win-Win

By: Andrew Goodman | Source: webpronews.com

Like a few others, I’ve been beta testing Google Website Optimizer. It’s great.

Here’s why it’s a win for Google. They’ll make more revenue because marketers who test will get impatient and need more data. They’ll enable their paid search accounts fully, possibly even enabling marginally performing groups, to get a statistically significant conclusion to their test more quickly so they can move on.

[via Search engine optimization and Online marketing. ]

I’m interested in Website Optimizer, too, though Google haven’t yet replied to my application for an account. I can’t say I’m quite as enthusiastic as Andrew Goodman, because Optimizer appears to solve only part of the split testing problem, making it a service that’ll be used mainly by a small sector of internet marketers.

The trouble is that to use it you have to paste code into your HTML pages. But most webmasters, especially those who are engaged in selling stuff online, don’t create their pages using plain ol’ vanilla HTML: instead, they have content management systems to do that for them.

The other problem with Optimizer is that it uses javascript to generate variant content dynamically. That content comes from a Google server. Rationally or not, many webmasters are not going to like relying on an external site to deliver parts of their precious pages for them. A selling headline that doesn’t appear because of a backbone router hiccup? No thanks!

For these reasons I don’t think Optimizer is going to wipe the floor with its rivals like Vertster and SiteSpect right away, even though Google’s offering is free in its basic form. A free service that’s very fiddly to use simply isn’t going to excite potential customers.

While we’re on the subject, take a look at my new blog about split testing. Whatever else Optimizer does in the market it’s bound to create something of a buzz around the subject, so I thought I’d try and catch some of it. Bzzz!