Measuring the Success of Interactive Marketing

Report on an NMK seminar in March 2002.

Speakers: Andy Flint, Head of Client Services, ABC Electronic; Simon Chamberlain, General Manager, Hitwise UK; William Makower, CEO, Panlogic UK

With more and more companies demanding to see tangible returns on investment in digital marketing, it is becoming increasingly important to be able to demonstrate the effectiveness of an online, email or wireless marketing campaign. But what exactly should be measuring, how should we be measuring it and how can metrics and statistics help us to achieve return on investment?

One of the advantages of digital interactive media is the ease with which data can be gathered about their usage. All websites generate log files that can be analysed by a wide variety of software tools or outsourced services such as Webtrends or Sawmill. The difficulty lies in deciding precisely what to analyse, and in determining the real significance of these data. Are 'page impressions' a more accurate metric than 'hits', for example, or should we forget them both and concentrate on 'conversion rates' and 'repeat visits'. The obvious answer is that it depends on what you're trying to find out.

Andy Flint is Head of Client Services at ABC Electronic, the digital media division of the Audit Bureau of Circulations (ABC), the industry body which provides data on the circulation and distribution of magazines, newspapers and directories in the UK and Ireland (thus enabling advertisers to make accurate judgements about the value and potential reach of ads placed in different publications). Andy began by explaining that he and his colleagues do not routinely calculate metrics for the sites that are monitored by ABC Electronic, as this can easily be carried out in-house by the site owners themselves. Rather, ABC Electronic is an audit firm which checks that companies are using methods that conform to the global common standards for comparison outlined by JICWEBS, the joint industry committee for web standards. (Similar principles are applied to audits of email and SMS activity.)

ABC Electronic also runs and associate subscriber scheme, which endorses log file analysis software and services that conforms to the JICWEBS guidelines (although the organisation always uses the raw data in a website's original log files when it carries out an audit, analysing these with it's own audit-command-language based package).

Andy and his colleagues favour quantitative research of websites, through a process of census counting that measures the total activity of an individual site each calendar month in terms of its:

• Unique users ('unique visitors' in US)
• Frequency of visits
• Page impressions ('page views' in US)

There is little point in gathering data unless you are also prepared to interpret it, and Andy believes that it is "fundamental to read only the useful data on the log files, the actual page requests". Log files actually record instances in which an external device communicates with the web server, and we infer that each of those devices corresponds to an individual user accessing the site. This is not necessarily true � one person might be viewing the site on two separate PCs, for example � which means that records of web traffic are always approximate.

Further inaccuracies can be generated by robotic activity (such as tickers), framesets, corporate firewalls that hide the IP numbers of unique users within a company, and ISPs that generate different dynamic IP numbers for account holders each time they access the net. As Andy pointed out, interpreting web metrics is not an exact science, but if everyone filters out inaccuracies where possible, and adopts common standards and methodologies, we will be competing on a level playing field. (For further details about ABC Electronic audits, JICWEB, and the associate subscriber scheme, visit www.abce.org.uk).

Analysing a website�s log files is not the only way of measuring the popularity of your online marketing spend. Some companies, such as Lemon Ad (www.lemonad.com), specialise in monitoring activity on the ad servers that provide advertisements to multiple sites, while other companies have developed their own techniques to provide different perspectives on web traffic metrics. One such company is Hitwise, which provides data comparing how clients� sites perform in relation to their competitors within particular market sectors.

Simon Chamberlain, General Manager of Hitwise in the UK (www.hitwise.co.uk, described how his company monitors internet service providers (ISPs), rather than individual web servers, in order to provide data on:

• Market share of visits (as opposed to unique users / reach)
• Market share of page impressions
• Average session duration

Data is based on a sample size of over 3 million internet subscribers (extrapolated to approximately 6 million users) across a range of ISPs, while over 150,000 sites are monitored within 152 pre-defined vertical markets or categories (e.g. Banks and Financial Institutions, IT Employment Agencies etc.). Sites are then ranked by market share within a category to enable clients to monitor the performance of an interactive campaign within the context of the overall market. According to Simon, monitoring traffic at the ISP level also brings other advantages, such as the ability to track users who navigate to a site from a bookmark or by typing in a URL manually � something that web server log files are unable to capture.

Simon outlined a number of case studies to illustrate how his services worked, including launch campaigns for X-Box and Opodo. In the latter instance, he was able to demonstrate not only how and when Opodo�s online and email marketing campaign had driven traffic to the Opodo site and increased its market share in the travel category, but also to compare it to competitors such as Ryanair.com. What this showed was that while traffic to Opodo increased sharply for the duration of the campaign, it dropped just as sharply as soon as the campaign ended, while competitors� traffic remained consistent � proving that �building a brand is not solved by bringing users to your site once�.

For William Makower, CEO of digital marketing agency Panlogic (www.panlogic.co.uk), the key to measuring return on investment in an interactive campaign is identifying what is worth measuring and what is superfluous. As the boss of JWP is reported to have said, �I know I spend 100% of my marketing budget, but I don�t know which 50% is effective�. It is very difficult to return an ROI on branding, for instance, as brand awareness is a very difficult thing to measure precisely. Figures for unique users, registrants, buyers and returning buyers, on the other hand, can tell you something more tangible, while the value of measurements such as clicks, ad impressions and search engine positions are debatable. Where search engines are concerned, the number of visitors coming to your site from the search engine is far more important than its ranking, while search terms are most illuminating of all.

In William�s view, measuring the number of users on your site can be less useful than analysing exactly what they do when they get there. Panlogic offers a service called �Route Path Analysis�, which provides a more sophisticated take on the common practice of measuring how long a user session lasts and what pages are visited during each session. Surprisingly, this system is not only able to break-down every single action taken by an individual user on a given website (including when transactions take place and how much money is spent), but to continue tracking all of their actions after they leave the site too (without contravening privacy regulations, as all personally identifiable information is cleansed by the user�s ISP). In this way, it is possible to monitor when your customers take their business to a competitor�s site, or to gain a much clearer picture of how your customers are using your services.

Using this system, Panlogic�s client Sporting Bet discovered how much more popular football is with its customers than racing, which had significant implications for the company�s business strategy.

In summary, William advised the following course of action:

• Identify your ROI
• Be consistent in your measurement requirements
• Demand accountability from your agency/in-house
• Review regularly and seek constant improvement

For the original event go here.