By David Hefendehl

Not even the most advanced analytics tools can uncover the ‘Why’ - you must instead turn to the customer. Of the many methods available to do this two are particularly helpful: testing and surveys.

Testing yields actionable insights swiftly but you are still only measuring how many people ‘vote with their feet’. Why they’re unhappy remains unknown, which is where open-ended surveys come in. Nothing else is better to get inside your customers’ heads.

A well-constructed and executed survey will finally give your CEO the answer to: “Why do they leave my homepage when we have this beautiful 10-minute Flash intro on it?”

The ultimate goal of any customer-centric business should be to tie together this sort of information with its analytics. By applying what you learn from the results of open-ended surveys with your analytics data you will get this 360 view of your customers.

Designing customer surveys brings further challenges, the single biggest issue always being, which questions will create the most valuable responses.

Try a simple exercise where you list all the questions that your superiors want to ask and with these ask yourself the following:

• Can I action the results of this question?

• Will this produce the most relevant data? Can you really do anything with data that says that 95% of your users love their mom? It might be better to ask if they would recommend the site.

• Is this question relevant to all of my customers? Questions that are obviously irrelevant may annoy users to the point where they abandon the survey. If someone has previously indicated that they are an employee then it might be best not to ask them how much time they spent on homework this week.

• Are we asking too many questions? It’s natural to want to include every possible question that you can think up in order to unlock all those customer secrets, but you risk scaring off the customer. Instead fewer, more carefully selected questions will give you just as good an outcome and without exasperating the user with endless questions.

• Is this question giving me data for the purpose of this survey? ‘Of course,’ you say, but is it really? When we present the results of particularly long surveys often essential questions were not answered. This is usually down to not asking the question in the right way or not asking the right question at all. There is no point asking if users received a print catalogue when you really want to know if they want to shop online on your site.

• Are you checking for task completion? Asking users how they rate a site is only partially useful. Users who have an abysmal experience tend to blame themselves and not the site’s IA if they manage to reach their goal. It might be better to ask: ‘Could you do XYZ’, or ‘Did you complete the purpose of your visit?’ Phrase this as an open-ended question and you will get pure consumer insight. Users will use this opportunity to praise or slate you - you will discover more than you thought possible.

By now you should have produced 5-10 questions that are essential to your business: questions that will produce results that will drive change and are actionable quickly.

The last task is to get a relevant sample. This is a minefield as everyone argues over what sample size is the most beneficial, but speaking to your survey provider is a good place to start. Exit surveys, such as the wonderfully easy to use 4Q, tend to have a much higher conversion rate than popup surveys that interrupt the user during their journey. We typically see a 2-4% conversion rate for pop-under (AKA ‘gently gently’) surveys and 0.25% for in your face pop-up surveys.

This will significantly impact how long you need to run the survey, how many people you need to invite and eventually the data quality. Expect to receive up to 10 times more complaints about “This Survey” when using a pop-up.

A good-sized sample is important in order to avoid acting on minority opinions. For instance, if you only get 10 results of your one million users and two of these had a problem finding the big red help button you could argue that 20% have difficulty finding it. If you get 30,000 in your result set of your one million users and 10 complain about the red button you won’t even see a spike in your data, allowing you to focus on the real issues such as your ‘illegible’ returns policy.

Act quickly on the big findings, and send the smaller ones straight into A/B testing. While you tackle the big problems – again, that important human touch – let automated testing resolve the smaller ones for you.

About Pod1

Based in London and New York independent creative digital agency Pod1 is a 50 strong collective of creative, strategic and technical specialists; all passionate about delivering cutting edge creativity.

Established in 2001, the agency creates highly effective ecommerce solutions for its clients, by investing time to understand a client’s business and combining this with exceptional consumer insight. Superior client service and planning underlie every campaign, the customer journey is fully explored, and conversion ultimately achieved.

Pod1’s specialist areas are retail, fashion, travel, and finance with expertise in web design, ecommerce development, Content Management Systems, Intranets and Extranets. The agency is also skilled in online marketing and online strategy solutions, with services including search engine marketing (SEO & PPC), banner advertising, email marketing and campaign micro sites.

Pod1 works with some of the UK’s and the world’s leading brands from an array of industries. They are the UK’s leading agency in the retail space and have an implicit understanding of this sector. Clients include: Marks and Spencer, Myla, Kurt Geiger, Uniqlo, Matthew Williamson, Burton, Jigsaw, Links of London, Barcelo, Kenwood, Net a Porter and Anya Hindmarch.

Founder of Pod1, Fadi Shuman, is also the official ecommerce blogger for Brand Republic and ecommerce expert for .Net magazine.

www.pod1.com