As an entrepreneur or product developer it’s often tempting to outline a roadmap for what’s technically possible or what we think is exciting and important. XKCD recognized several years ago that people have been building what they think is nifty or even necessary, without researching and identifying the value adding features users are looking for.
MailChimp even wrote about this tendency to develop the next big thing in their blog ”Here are 32 Seconds, Make the Most of ‘Em.” They had been tweaking the UI and adding new campaign options which were supposed to make it easier and faster for customers to send out emails. They realized they could actually test this through a bit of research. They got scrappy and had current employees try out ‘b’ versions of the webpage while another group tried the existing ‘a’ version. The metric they tracaked was number of seconds required from page load to sending a campaign. One might argue that this is a vanity metric and should be ignored, assuming they don’t receive additional revenue by lowering time to complete campaigns. However, if you have found that users are taking off for your competitors because it takes too long to send campaigns, you just might be on to something.
After their first round of research, they dug a little deeper and got actual MailChimp users to try both versions of the site. They found that users were able to create new campaigns 32 seconds faster with their new UI. Improvement!
The lesson? Let your users tell you what they like and don’t like with a bit of testing. You can let your qualitative testing guide your strategy for future features, but don’t spend a lot of capital building things out until you know it’s actually providing value.