I had the pleasure of attending the RefreshDC presentation on Data-Driven Design that was given in May, 2011 and it really opened my eyes up to some of the great tools there to collect concrete data on how users are using your website. What they are clicking on, where their focus is, how far down the page they are scrolling, and just how much of your site they are actually seeing. Also, if that button to get them to download your application was just a bit bigger, just a bit brighter, or labelled just a bit differently…would it matter?
This particular meeting was set up as a panel discussion in which Greg Robletto, The Motley Fool’s Director of User Experience moderated over a small panel that was comprised of:
- Paul Koch, Marketing Specialist at Viget Labs
- Jim Lane, Director of user Experience for AddThis.com at Clearspring Technologies
- Craig Green, Web Strategist at Network Solutions
The talk started out discussing a bit about Google Analytics, a crowd favorite that is well known by most people who work on the web, but then really focused in on various tools that are available for A/B Testing to allow web professionals to try out different elements, styles, and look and feels for a particular site, and gain statistical information on what users did or did not respond to. But so we all know about the wonder of Google Analytics, and the rich information that it can provide. We also know how to swap out controls and segment our users to see what works best – but do you know of all the tools out there to help you with A/B testing, and to help you collect actionable data from these tests?
So what other tools are out there?
Once the discussion got going, there was a bit of a free-for-all where various tools for data on your website was thrown out there and discussed. Given that many of these tools are new to me, and not necessarily my area of focus, in some ways it would be a disservice for me to try and describe what they do and how well they work at their jobs. I think what may be more helpful is for me to point out some of the tools that were mentioned to allow you do do your own research on them, and go from there. To me, this is part of the job of helping you to tackle the question of “How do you know what you don’t know?”
Take the following descriptions as you will, I’ve chosen in most cases to use the marketing lingo that each of them has on their website:
- Ninja buttons – Creae, test and optimize call-to-action buttons to convert more visitors inot paying customers. Allows for A/B testing of various buttons.
- Omniture – helps marketers quickly identify the most profitable paths through a website, segment traffic to spot high-value web visitors, determine where visitors are navigating away from the site, and identify critical success metrics for online marketing campaigns
- Google Analytics – A crowd favorite, I don’t think I need to describe all the wonderful things that this tool does
- UserTesting.com – The fastest, cheapeast way to find out why users leave your website
- SurveyMonkey – SurveyMonkey makes it easy for you to create you own Web-based surveys
- Google Website Optimizer” – Instead of guessing what your visitors respond to, get hard data on which images, headlines, descriptions, and layouts produce the outcomes you’re looking for. There was some discussion on how the data out of the A/B testing is not accurate.
- BTBuckets – allows websites to increase engagement and ultimately maximize conversion rate optimization in real-time, by segmenting and targeting strategic user groups.
Many of these tools seemed great, and were discussed by the panel, but really digging into them was left for the audience to do. One of the tools that was presented by Viget’s Paul Koch was ClickTale, and the benefits of this tool were readily visible.
Through the use of heatmaps that track where users move their mouse to, where these scroll to, and where they click on, ClickTale allows you to ‘watch’ all the users on your website without standing over their shoulders. It is well known that users tend to move their pointers to where their eyes are focusing on a page, and through these heatmaps, it is apparent that this hypothesis is true. Of course the question came up, but what about mobile? That remained an unanswered question for this panel.
But so now that you have tools to help you gather data on how your users are using your website and tools to conduct A/B testing on your site, should you just blindly follow the data? As designers and developers it isn’t just your technical abilities that make you valuable, but also your ability to reason through things and sometimes rely on your intuition. The data may be guiding you in a particular direction, but your expertise may tugging you in just the opposite way.
Not listening to the data – when do we want to do this?
I think that Paul put it best with the following:
“Even if a big ugly BUY button drives traffic if it detracts from the brand you may not want to do this.”
And this was backed up a bit by a real world examples where an online survey of the various widgets available for AddThis had the widget on a black background as the one that was most liked and stood out.
ClearSpring’s Jim Lane said that the AddThis UI gets rendered about 2.5 billion times a day. Even though the black background version won the A/B test it doesn’t mesh well with many sites and thus isn’t the default UI. That makes great sense when we think about websites that are out there, but is a great example of when we choose not to necessarily listen to the data.
But often they have listened to the data, as in their constant effort to make it easier for end users to download their widget. By gathering data from the user, they chose to remove steps/clicks to get to the AddThis code and increased usage by a measurable factor. Other testing that they have done includes a 32-way test on the “Grab This” button. different sizes, Colors, etc. Lucky for them with 2.5 billion renders a day, their userbase is large enough to get statistically significant data on such a large A/B test in under a week.
So the goal of this post is to arm you with various tools and methods to collect data on how your users are using your site, and what changes work best to attract your users. Hopefully these tools help you determine what keeps them on your page, but the question still remains – how quickly you should make changes to your websites?
Do you do small changes in order not to throw off your regular user base, or do you lump all the changes in at once, and just throw in out there. Here are some thoughts:
“How quick to make a change to the User Experience for testing? A bit of an Art. Sometimes you’ve got to go large.”
“To get statistical data on a lot of changes can take a lot of time so sometimes it is easier to take a change at a time. – Paul”
But one strong word of caution came from the audience that I thought was especially relevant and may help you figure out how you may want to time out your changes:
“You have to be careful – working with these tools can get addicting and you can do tweaks forever and meanwhile the business is moving on.”
Optimizely
At the end of the session, one of the panelists tool one of the tools he had just heard about for a test run. The tool was Optimizely.com and it was pretty amazing how with just their “Get Started” tool on their front page, we were able to redesign the CNN website in just minutes. You should head out there and give it a try.
Staying Ahead of the Game
But so now that you are armed with various tools to get you stared down the path of data-driven design, how do you stay up to date on the changing world? How do you know when new tools are out there, and when some existing tools may not provide the most adequate data? I love the fact that the panel discussion ended with resources on how to stay up-to-date in this field. In fact, if I’m not mistaken, I believe Paul said that he found out about Optimizely.com from the first reference in the list:
And Jim was kind enough to provide his twitter handle, where he also adds information on Data-drive Design-@jimlanenova
Thanks to Viget Labs for hosting RefreshDC for the month, and OPower for being the food sponsor, and of course the great people who donate their time to keep RefreshDC going!
