Service Management

4. Benchmarking - 2
Forbes Gibb
forbes.gibb@cis.strath.ac.uk
Objectives
  To review measures and metrics for class
exercise
  To identify potential benchmarking sources
Benchmarking Exercise

  Consider a web-based ordering system for

IT products, consumables, etc.
  Mixture of quantitative and qualitative
measures
  What should you measure and what
metrics would you use?
  How would you compare?
Benchmarking Exercise
Group Consensus
  Visitor conversion rates
  Availability of site

  Reliability of site

  Security effectiveness

  Order efficiency
Group Consensus
  Registrations
  Server performance

  Site administration costs

  Customer satisfaction

  Number of visits
Key Phases of Online
Commerce

  Acquisition – the customer finds the site

  Exploration – the customer looks for products

  Conversion – the customer buys products

  Retention – the customer locks onto the site

Acquisition - 1
  Visibility:
  Search engine visibility (rank per term; rank per brand
name; rank per company name)
  Overall visibility (rank per term; rank per brand name;
rank per company name)
  Advert effectiveness:
  Advert mouse-over rate (mouse-overs / deliveries)
  Advert click-through rate (click-throughs / deliveries)
  Brand impact (recall percentage)
  Cost-effectiveness (£ per click)
Acquisition - 2
  Reputation indices:
  Reviews (rating per product; rating per service)
  Complaints (number of negative comments –
www.sucks500.com)
  Visits:
  Hits (number per hour/day for home page, entire site, etc.)
  Raw visitor sessions (number per day, per hour, etc.)
  Average (average number of visitor sessions per day each
reporting period)
  Specific visitor sessions (number of international, national,
unknown, unique, qualified, etc., visitors)
  Cost effectiveness (average cost per visit per reporting period)
Exploration
  Session characteristics
  Search terms (frequency per term)
  Pages consulted (number per session)
  Entry page (frequency per entry page)
  Exit page (frequency per exit page)
  Most / least popular pages (frequency per page)
  Pogo-sticking (number, %age of repeated link
activation)
  Stickiness (total amount of visit time / number of
unique visitors)
Conversion
  Conversion rates (%age of visitors)
  Value (average £ per sales)
  Volume (number of sales)
  Average transaction times (seconds per transaction)
  Abandonments (volume, or £ lost; %age of starts)
  Clicks to complete (average number of clicks per
transaction)
  Cost per sale (click through costs : sales value)
  Customer satisfaction (%age of returns)
Retention
  Registrations (volume, %age of visitors)
  E-mails/enquiries (volume, %age of visitors)
  Downloads (volume, %age of visitors)
  Repeat business (volume, %age of visitors; £ for
repeat sales : £ for new sales)
  Return visitors (volume, %age of visitors)
  Return customers (volume, %age of visitors)
  Mean time between return (average number of
days)
Technology Performance
  Latency
  Discovery times (turns)
  Transfer times (payload)
  Search times (seconds per query)
  Reliability
  Failures (volume per week/month)
  Mean time of failure (seconds per failure)
  Meant time between failure (days between failures)
  Availability
  Uptime (%age per reporting period)
Functionality and Features

  Order tracking
  Cross-selling/referrals
  Secure transactions
  Price comparisons
  User accounts
  Search engine
  Finance
  Personalisation/localisation
  Platform independence
  Compliance: DDA, DPA, WEEE, etc.
Other Measures

  Customer satisfaction (Scales, ranks):

  Efficiency, effectiveness, utility
  Usability (Scales, ranks)
  Data consistency (%age)

  Data concurrency (%age)

  Findability (%age)
E-Commerce Benchmarking
  Clicks easier than bricks
  More accessible
  More visible
  Customer driven
  Mystery shopper
Sources of Data
  Web server log files
  Record of page requests
  Cookies
  Parcels of text sent to the user (client) by the
server
  ID=40dbd37914242a34:TM=1013725751:LM=10
13725751:S=P4MUPnk7Wbs
  Third parties
  Collect and analyse data
The Importance of Log Files
  Who made a request (IP address ; username)
  When a request was made (including time zone)
  What was requested
  How the request was made
  Whether the request was granted
  How much information was sent
  Where the request came from (referrer)
  What software they used to make the request
  What they were looking for (including search terms)
  How long they stayed
  Which entry and exit point they used
The Importance of Cookies

  Authentication
  Session tracking

  User information and preferences

  Network Advertising Initiative

  http://www.networkadvertising.org/index.asp
Network Advertising Initiative
Remove Cookies on your
Machine

1164
cookies
Third Party Web Analytics
  http://wordtracker.com (visibility)
  http://www.dynamiclogic.com (advert
effectiveness)
  http://www.submitawebsite.com/(reputation)
  http://www.webtrends.com (visitor intelligence)
  http://www.google.co.uk/services (Google
performance
  http://www.gomez.com (application
performance)
Possible Benchmarking Sources
  Public accountability data and national
statistics
  UK Department of Business, Information and
Skills
  Tripartite Authorities Business Continuity
Benchmarking
  Public Sector Benchmarking Service
  DG Enterprise and Industry
  Consultancy firms (third party databases)
IT/IS Benchmarking
  Forrester Research TechRankings
  Standard Performance Evaluation Corporation (mail
servers, to power and virtualisation)
  Ideas International Competitive Profiles
Benchmarking Gateway
  Storage Performance Council
  NSS Group (security)
  Transaction Processing Performance Council (DBMS)
  Benchmarking Network Inc (ABC, to help-desks and
total cost of ownership)
Distance Selling Regulations
  Distance Selling Regulations (2005) give protection
in EU to consumers who shop by phone, mail order,
via the Internet or digital TV including:
  The right to receive clear information about goods and
services before deciding to buy
  Confirmation of this information in writing
  A cooling off period of seven working days in which the
consumer can withdraw from the contract
  Protection from credit card fraud.
  The Financial Services (Distance Marketing)
Regulations 2004
Invitation to Treat
  An invitation to treat is an action inviting
other parties to make an offer to form a
contract
  Argos mistakenly offered televisions online
for £3.00 instead of £299.99
  They then refused to fulfil orders at that price
insisting that no contract had been formed
and that they were “subject to availability”
Summary
  There can be many possible measures and
metrics
  Essential to have a clear idea of the
objectives of the exercise and match these
with the measures and metrics
  There are also many sources of comparative
data, some free, some commercial