ISBM Business Market Research Consortium Nuggets

Gathering Primary Data on the Web

How to research your markets quickly and efficiently via the Internet.
A key points summary from the fourth meeting of the ISBM Business Market Research Consortium

October 30, 2000 Philadelphia, Pennsylvania by

Bob Donath Bob Donath & Co., Inc.

Contents:

Open mike roundtable by consortium members. Featured Practice Overview: Allen Hogg, Burke Interactive, Conducting Primary Business-to-Business Research Via the Internet Featured Practice Overview: Karl Feld, humanvoice, Inc. e-Interviewers: Adding the Interviewer Into Web-based Research. Featured Research Overview: Arvind Rangaswamy, Penn State. Rethinking Marketing Research for the Digital Environment.

2000 ISBM, Penn State Smeal: Business at Penn State

OPEN MIKE ROUNDTABLE ON MEMBER ISSUES:

A summary of business market online research management observations and challenges cited by meeting participants. Researching the value chain We need to look down the value chain of the packaging industry for primary and secondary research on consumer food consumption trends. We hope to learn how to do that at this meeting. (plastics manufacturer strategic planning manager) We expect to run tests and see if online research actually reduces the cost and increases the speed of looking downstream to end users, compared to traditional research methods. (chemical company product commercialization manager) New product development We need primary research to guide research and development, and new product commercialization. (bulk chemical company information manager) Im here to learn whether we can use online research as an early stage strategic screen for new product and plant investments. (petrochemical company market research manager) The fuzzy front end of the new product development process is a major concern we hope to research online. (chemical company new business development director) We expect Web-based research can help us extend our product planning horizon by several years, as well as improve service for existing customers. (petrochemical strategic research manager) Learning to use a new research medium Although Im an experienced researcher, Im always seeking new ideas, particularly about using the fast-developing Internet. (bulk chemicals company technical specialist) We seek new ways to help our clients conduct online market research. (research company partner) Were uncertain how far to take online market research, both externally and internally via the corporate intranet. (information systems company market research manager) We are always on the lookout for new research tools available for our internal market research staff as well as those available from external suppliers. (industry information and retailing portal product development director) Intranets are an excellent way to distribute market research findings across the corporation. We refer inquiries to the data on our central site. (specialty chemical company research director) Our three goals for online research are measuring brand advertising ROI, conducting surveys, and developing a knowledge base for information sharing across the corporation. (diversified technology equipment manufacturer corporate brand manager) How can we use the Web to save money and still get representative survey samples for our voice-of-the-customer studies? (business equipment manufacturer market research manager) Comparing online research to traditional methods

 Weve found so far that online market research is not much cheaper than using traditional methods, but we keep looking for good Internet research applications. (plastic packaging materials market research manager) Our business unit has not conducted online research yet, although other company units have. We plan to start with a value and loyalty study, comparing online to traditional methods. (plastics manufacturer eBusiness manager) Online focus groups look promising for researching some of our target markets, but we are concerned about the representativeness of online survey samples. (specialty chemical manufacturer market research manager) We use the Web for primary and secondary research of new markets. The Web allows more of a global information search, but the information it provides has holes we need to fill with traditional methods, such as phone calls. We also rely heavily on our salespeople for customer feedback. (specialty chemical company market research manager) Improve customer communication Were looking out for better techniques running customer surveys and discussions online. (information systems company product director)
Return to the Table of Contents

Featured Practice Overview:

Conducting Primary Business-to-Business Research Via The Internet

Allen Hogg
Director of Marketing Burke Interactive Ridgefield, Connecticut

The Growth of Internet Marketing Research

Market research on the Internet, growing an estimated 152% this year according to Inside Research, has exceeded forecasts over the past two years. Inside Research data are based on the spending of 29 research companies accounting for an estimated 90% of U.S. online market research. 83% is replacement research: Traditional or replacement research with data formerly collected by phone, mall, etc., but now replaced by online, plus new research methods only executable online. Concept screening, concept/product testing Advertising/brand tracking Customer satisfaction measurement Employee surveys Online focus groups 17% is Web-centric research: Web site measurement and evaluation. Profiling Web-site visitors

Web-site evaluation Banner ad testing Conducting Internet research can be Faster In 1998 study, Andrew Findlater, Reed Business Information received 160 of 400 required interviews in three hours Harris Interactive: 70%-80% of completions occur in the first 48 hours Generally, a majority of completions will occur within the first 48 hours Cautions: B2B research not quite as fast Do not stop interviewing just because quota has been met. Provide sampled respondents enough timeat least a weekto complete the survey before suspending data collection. Early respondents to email invitations might not be representative. Less expensive. The marginal costs of additional survey responses can be less than they are for mail or telephone. Incentives, however, can lessen or eliminate any cost advantages (Exhibit 1). Exhibit 1

More enjoyable for respondents.

89% with Internet access prefer to receive any survey via the Internet, according to Michael Foytik of DSS Research. White paper available at http://www.dssresearch.com/resources/bibliography.asp A study by Burke Interactive found greater likelihood of respondent participation in the same study in future research 33% of Web survey respondents saying they definitely would, 75% saying probably or definitely would. Just 18% of phone survey respondents saying they definitely would, 62% saying probably or definitely would. The survey, with limited open-ended questioning, took 12.4 minutes for average completion on the Web; 19.5 minutes by phone. [Comment by Arvind Rangaswamy of Penn State: There is some evidence that too many surveys on the Web are making the medium less productive, prompting some companies to return to phone surveys.] The trouble with traditional methods. Increasing non-response error. Phone refusal rates up, from 40% in 1988 to 46% in 1997, with no sign the trend is abating, according to The Council for Marketing and Opinion Research Mail response down. The Census Bureau assumes that about four out of 10 U.S. households that receive a 2000 census form wont fill it out, even though they are legally required to do so, American Demographics reported. Mall intercept completion rates down from 30 per day per mall in the 1970s to fewer than 5 in the 1990s, according to A.C. Nielsen. [Comment by Ada Nielsen of BP Amoco: Internet surveys are not as anonymous as mail surveys can be. Another problem is that Internet surveys work fine for business respondents using high-speed office connections, but are less satisfying if done at home over dial-up modems.] [Response by Allen Hogg: We have found there is little response difference between those who give their name online and those respondents who do not.]

How to generate an Internet sample

Panels Recruited from many sources Portal relationships Banner ads Phone and/or mail and/or existing mail panels List brokers Contacted frequently Some panels will survey a panelist every five days. Risks developing professional respondents. We contact panelists monthly. Companies attempt to forge ongoing relationships with respondents. Survey restrictions apply Typically have a fairly good demographic profile on file

Considered the Gold Standard for Internet respondents E-mail lists / databases Recruited from fewer sources Almost always recruited online, typically from a Web site People typically answer affirmatively that would be willing to be re-contacted for other research Contacted less frequently. Some panelists do not know they are in a database Less of a relationship. Survey cooperation rates can be much lower. A potentially good source for particularly targeted samples; e.g., diabetes sufferers Site-centric intercepts Typically used for Web-site evaluation, but could be used for any type of survey where Web-site visitors are the sample population Passive Recruiting (not recommended) Done primarily with the use of click me and interview me buttons. Drawback: respondent self-selection, akin to customer comment cards Benefit: less for the Webmaster to worry about Active recruiting (random or sequential) Done with behind-the-scenes software that is similar to Random Digit Dialing (RDD) techniques. Benefit: more representative ending sample Drawback: initially more work for the Webmaster Actively pulls aside every Nth visitor Cookies prevent respondents from being chosen more than once Able to sample visitors as they enter a site, exit a site, or both Response rates have run as high as 65% (avg. 25%) [Audience member comment: the technique described is more like interval sample than true randomization of dialing digits from a known database.] Customer lists / client databases / employee databases Merge and purge duplicate listings Make random selection from unduplicated list Send outbound e-mail invite with unique user ID embedded NEVER, NEVER, EVER include many addresses in a visible cc outbound e-mail Include URL (Uniform Resource Locator) that can be clicked on within email or cut-and-pasted into a browser. Many organizations are creating their own proprietary customer/industry interactive panels Ask about Internet usage during all research Ask respondents about their willingness to participate in future research Collect e-mail addresses of individuals interested in being included. Make access to topline information part of the incentive. Banner ads (not recommended) Self-selection bias.

 Unknown population SPAM (not recommended!) Phone/mail recruitment to Web (not recommended) Weve tried, but response rates are low. [Acknowledging audience questions: We have not tried inexpensive postcard mailings. Nor have we tried recruiting at trade shows, but that might work.] [Comment by Karla Kuzawinski, independent research expert formerly at Xerox Corp.: We received a good response in B2B mailings, where customers clearly see the benefit of responding.] {Comment by Karl Feld, Humanvoice: We tried a mixed mode survey recruitment using mail and email, and found no difference. However, it was with a universe of loyal individuals, university alumni.] Mall intercepts, a consumer market option.

Programming and designing an Internet survey

Surveying tools HTML Surveys using Hypertext Markup Language, the non-proprietary standard programming language for creating linked World Wide Web documents. Can have complex logic (skips, terms), data piping, and rotation/randomization of questions and responses Can expose respondents to visuals Software for conducting conjoint and discrete choice studies online has been developed Numerous off-the-shelf do-it-yourself packages available. Email Surveys Software can parse responses from return e-mails Not capable of skip patterns, rotation/randomization Java Surveys Java technology allows interactive applets to run on numerous computer platforms Developed by Sun Microsystems, Inc. Incorporated into all major World Wide Web browsers, but still can create problems. Method really hasnt been widely adopted. Must customize program Principles for Web survey construction (from Dillman, Don A.; Robert D. Tortora, and Dennis Bowker, Principles for Constructing Web Surveys, 1998. Social and Economic Sciences Research Center Technical Report 98-50, Washington State University, available at http://survey.sesrc.wsu.edu/dillman/papers/websurveyppr.pdf) 1. Use a welcome screen 2. Have a good first question. The first question is the biggest respondent drop-off point. 3. Use a question format that is similar to a paper questionnaire 4. Beware of text line length exceeding the width of the respondents browser

5. Provide specific instructions for any computer actions that may be necessary to answer a question 6. Put computer instructions with the question where the action is to be taken 7. Use caution when requiring respondents to provide an answer to each question before being allowed to proceed to the next question. Only do it if the answer determines question skip patterns. 8. Balance the use of a single scrollable questionnaire with a one question per page questionnaire 9. Be careful with questions with a large number of response alternatives. Be sure respondent can view all response options at the same time, on the same screen. 10. Let respondents know where they are in the questionnaire 11. Choose question types carefully and avoid problem questions. We found open-end question response better on the Web than in phone surveys. Offering a dont know option does not improve completion rates, we found. Additional best-practice suggestions: Allow sufficient time for responses; give all sampled individuals the opportunity to participate. Leave the response window open at least a week. Incentives and drawings can boost response rates, but be careful about violating lottery laws. Knowing the law is essential for international surveys Dont underestimate the importance of technical support; give respondents e-mail addresses and phone numbers to use if they encounter trouble. Try to use an FAQ link from the start of the survey to handle simple issues. Test the interview on as many browser/operating system/ platform combinations as possible. This is less critical a problem nowadays. Figure out how long it will take respondents using standard speed modems, and be honest about the length. Keep the interview short. Make user IDs and passwords good for one pass through survey only. Recognize that concept descriptions and graphic images should never be considered completely secure. Put your privacy policy statement on the questionnaire. Predicted respondent dropout rates. (Exhibits 2 through 5).

Exhibit 2

Exhibit 3

Exhibit 4

Exhibit 5

Pay attention to privacy concerns! Numerous organizations are policing the Internet regarding privacy Mail Abuse Prevention System (MAPS) LLC PrivacyPlace TRUSTe Online Privacy Alliance (OPA) Platform for Privacy Preferences (P3P) Electronic Frontier Foundation Better Business Bureau OnLine CPA WebTrust Governments are also interested U.S. Congress and the FTC European Union Directive on Data Protection Marketing research organizations have put forth ethical standards for Internet use; check their policies AMA (http://www.ama.org/about/ama/ethcode.asp) ESOMAR (http://www.esomar.nl/guidelines/internet_guidelines.htm) IMRO (Interactive Marketing Research Organization) http://www.imro.org/code.htm

CASRO (Council of American Survey Research Organizations) http://www.casro.org [Meeting attendee comments: incentives that have been tried frequent flyer miles computer printers charity donations (particularly important when government employees cannot accept gift) Amazon.com gift certificates Freestuff.com can draw freebie-hunting consumers to your survey, with surprising results. One chemical company doing a plastics study and giving away potholders to B2B respondents had to fulfill 60,000 for one days responses triggered by a Freestuff posting].

Is it valid research?
The Internet research peril: Biased data collected in vast amounts has absolutely no benefit to your organization! Sources of survey error: Sampling Error: sampling only a fraction of the population Coverage Error: excluding population segments from drawn sample Non-response Error: sampled individuals do not complete the questionnaire Measurement Error: a result of question wording or mode effects. Respondents might respond to scales differently on phone and Web surveys, for example. In many cases, all-Internet study designs introduce coverage error by excluding important parts of the population to be studied from participation. Although Internet penetration is increasing, many parts of the population including many companies customers and employees remain offline. Over-reliance on the large samples typically seen in Internet research can give researchers a false sense of confidence. It is quite possible that the one can precisely measure the wrong thing. Comparisons of survey methods: Comparing online, random digit dialing, and mail panel sample representativeness, Web respondents tend to be more educated, with larger incomes (Exhibit 6). Web respondents are more technically savvy, and use technology more. Demographic weighting probably cannot eliminate the discrepancy (Exhibit 7). Do not assume that an online survey response will be representative of online users. An online survey will likely not even generate a sample of Internet users similar to Internet users reached through traditional research methods (Exhibit 8).

Exhibit 6

Exhibit 7

Exhibit 8

Remember that connected individuals are still a small minority worldwide.

Mix online with other survey methods?

B2B is a special case. Business-to-business companies are becoming especially concerned about harassing their valuable clients in order to get important feedback from them. A multi-mode data collection design makes it possible for more people to respond to a survey in the way that is most convenient for them. Multi-mode research Multi-method/multi-mode data collection is not the same as hybrid methods current in use Mall phone Phone mail Phone mail phone Multi-mode research is using two or more methods for the same survey. Potential applications: Use different data collection methods in different countries Use phone or mail to interview non-connected individuals to add to an Internet sample Allow the respondent the choice of methods First attempt to survey respondents through the Internet, then backfill with phone interviewing Potential benefits and drawbacks Potential benefits Added convenience for the respondent Can use less of respondents time Better management of the customer relationship

 Typically faster cycle times Typically a lower cost Enhanced ability to collect data across the world Switching modes can enhance the perceived importance of the survey and increase response rate Lower non-response error! Better enables Web reporting Potential drawbacks Potentially higher programming costs Greater coordination between sample control for the various methods is needed Greater attention to scaling and study design is needed Potentially higher Measurement Error: The result of inaccurate answers to questions that stem from poor question wording, poor interviewing, survey mode effects, and/or the answering behavior of the respondent (Dillman, op.cit.) Methodological differences (Exhibit 9). Phone seems to exhibit a recency effect while Web questionnaires, showing all response options, get more of a neutral response. Exhibit 9

Survey design implications. We recommend completely labeled and numbered scales (Exhibit 10).

Exhibit 10

Objectives: Maximize coverage Reduce non-response Minimize measurement differences No calibration of data needed Scales need to work well with both phone and Web administration Generally use completely labeled scales only when necessary (e.g., satisfaction) Avoid scales with seven or more scale points that are verbally labeled use end-point anchors instead. Construct surveys with eye toward unimode (Dillman, op.cit.) design (Exhibit 11). Note that the example does not permit a numerical scale. Create common question text Avoid the use of check all that apply type questions Avoid the use of drop-down boxes Avoid visual metaphors of scales Avoid the use of a single scrollable screen Use Web technology that can emulate CATI logic control (i.e., skip patterns should be passive) Email surveys might not be a good multi-mode choice

Exhibit 11

Reporting survey findings on the Internet

Evolution from paper reports. Putting Word or PowerPoint files on a shared network Having an Intranet page with a link to these files An Intranet is a private network utilizing Internet software and protocols It can also be used for conducting surveys of employees Producing documents in Word or PowerPoint, then saving as HTML for Intranet posting Creating hyperlinked reports using HTML-authoring program Using Online Analytical Processing (OLAP) applications Enables real-time, on the fly access to collected data via an interactive WWW interface Drill down, drill up, slice and dice Online reporting considerations Static versus Dynamic: Static reports permanently represent the results in a pre-formatted manner. A competent researcher decides the appropriate views and perspectives on the data with no manipulation possible by the research users Dynamic reports allow for interactive content manipulation, allowing for flexible interpretation. Enables individuals to create other perspectives such as aggregation levels, filters & sub-samples. Discrete, near real-time & real-time: Discrete reporting captures results for a fixed time period (i.e. a week, a month, a year)

 Near real-time allows for continuous data with a minor lag for automated data upload (i.e. results updated every day at midnight) Real-time allows for up to the minute changes in results (i.e. the last completed interview is available upon any request) [Comment by Karla Kuzawinski, independent research expert formerly at Xerox Corp.: Real-time reporting also provides an early warning system for survey problems that might arise.]

Qualitative research via online focus groups?

Hybrid application Face-to-face focus group similarities For research purposes Intent to learn on behalf of a sponsoring client Moderator Respondents Revealed discussion topics Process dynamic Timed meeting Recorded Online Chat Similarities Underlying technology is borrowed from chat software. Options include IRC, ichat, itracks, Uplink, et. al. All communication is text submitted by individual respondents. [Audience comment: Groupwise (http://www.novell.com/products/groupwise/) provides better-than-a-chat-room focus group environments. Ventana Systems software (http://www.vensim.com/) is a powerful tool for similar collaboration Online focus groups arent right for every study Lose immediate reactions, tone of voice No visual contact Non-auditory Only generate 75% as much content in same amount of time Comments are made slowly but surely Need to be prepared for slowness of the burst process Works better for groups of people with a common level of shared experience Participants can be more easily distracted Participants need to know how to use software Every cool feature disenfranchises a few more potential participants But advantages are clear Bring together geographically dispersed groups, even internationally. Provides a larger population for recruiting No travel for moderators or clients, either. Other stakeholders can observe from their desks or homes.

Considered, precise, unembarrassed comments. A more natural setting for some (e.g., software developers). Equal air time available for all participants. Circumvent problems caused by group dominators. Can incorporate powerful qual/quant hybrids Can gauge reactions to multimedia stimuli on World Wide Web. Frequently used for Web-site evaluation research. Immediate electronic transcripts are natural by-product. Every comment can be associated with a writer and the time when it was written. Verbatims can be easily inserted into electronic reports, posted on Intranet sites. Preparation Screen for chat room experience Have people test out access ahead of time Work to avoid losing respondents due to distraction. Remind them that incentives are available only to those who stick with it. Create detailed discussion guide from which you can cut-and-paste. Questions to consider when choosing software and suppliers. Will the focus group site be secure? Are participants given user names and passwords? Will participants need to download any software? Is it possible to establish private channels? Can you send private messages to the moderator? Can the moderator send private messages to individual participants? Can you throw out problem participants? As for online research partners: Are they good researchers? Do they provide research solutions, or want you to buy Internet research? Online experience is necessary, but not sufficient. Can they provide data from multiple streams (e.g., Web, phone, IVR, etc.)? Are they informed about the differences between methods and can they help you expect to see certain effects? Can they add value to the data through analysis? Do they respect privacy issues and will they support industry standards? How do they provide sample (intercept, recruit, panel, etc.)? Are they versed in the issues of online sampling? Where will the data reside, and is it secure? What are their interviewing capabilities? Similar to CATI, or limited to simple questions? Is their interviewing platform stable (24/7)? Can they provide research results online? Consider the Internet buzzwords of 2000 Broadband

 Mainly now ISDN (integrated services digital network), DSL (digital subscriber line), and cable modem Billions now being spent on broadband satellite systems 11% of U.S. households with Internet access now have such broadband connections (Statistical Research Inc.) Enables online testing of commercials and other multimedia products, but Coverage error concerns of general online research can be magnified Wireless By next year, 1.3% of U.S. residentsand 8% of Western Europeanswill be using wireless Internet services (Forrester Research) Numerous companies are working on survey technology for personal handheld devices Potential for survey respondents to be using very small screens reinforces the notion that questionnaires should be designed simply
Return to the Table of Contents

Featured Practice Overview:

e-Interviewers: Adding the Interviewer Into Web-Based Research

Karl G. Feld
Vice President of Research Services Humanvoice Inc. Provo, Utah

Importance of the dyadic

Interviewers, the people part of the survey experience, improve data quality. Bourque, L.B. and E. P. Fielder. 1995. How To Conduct Self-Administered and Mail Surveys. Thousand Oaks, CA: Sage Publications, Inc Conrad, F.G. and M. F. Schober. 2000. Clarifying Question Meaning In A Household Telephone Survey. Public Opinion Quarterly. 64:1-28. Krosnick, J. 2000. Peering Into The Future of Thinking and Answering: A Psychological Perspective on Internet Survey Respondents. Presentation at the 2000 Gallup Nebraska Symposium on Survey Research. 13-15 April. Lincoln, Nebraska. de Leeuw, E. and M. Collins. 1997. Data Collection Methods and Survey Quality: An Overview. Survey Measurement and Process Quality. New York: John Wiley & Sons, Inc. Saris, W. E. 1991. Computer-Assisted Interviewing. Newbury Park, CA: Sage Publications. Use e-Interviewers to: Reduce mid-termination or dropout rates significantly. Increase open-end response length. Screen out ineligible and professional respondents. It is inexpensive. Technologies control data quality across modes Integrated CATI and web survey software to allowed uniform, simultaneous programming. (You cannot administer phone and Web surveys identically.) List Rotations Complex Skip Patterns Forced Responses Prevent Order Effects Respondent Privacy (Todays top online market research issue) Internet versions hosted in-house to guarantee respondents server access 24/7. Servers were tied directly to the Internet through a Tier-1 provider. Use a researcher tied directly to the Internet, not one linked through an ISP. Servers were redundant and load-balanced to prevent overload during peak periods. Good online research firms will provide this. The humanvoice screen presented to sample respondents features: The question (1 or 2 per page) and response options.

Text chat with interviewer, a live person watching the respondents progress through questionnaire. The interviewer might send encouraging messages as appropriate. Click option for respondent to ask interviewer to call on telephone, for help, etc.. (The respondent can designate when the interviewer should call.) The interviewers picture to humanize the contact. Demonstration and white paper available at www.surveyguardian.com

Comparative research findings by humanvoice

Quotas based on winter 2000 U.S. Internet population. Age Census region Gender Income Occupation Sampling from known universe CATI version Entire known universe (141,337 names) used. Selected every nth point based on geographic, gender and age quotas. Internet versions Those with e-mail addresses not contacted by telephone (11,892 names) used. Separated into demographically equivalent files, one for each version. Every nth name selected based on geographic, gender and age quotas. Data collection procedures By CATI Interviewer hours evenly divided by gender. No interviewer worked two days sequentially. Daily quotas by quota area using sample proportional to final quotas. Dialed sample from 9 AM to 8 PM after initial evening attempt. Callbacks were scheduled upon request. By e-interviewer Web Interviewer hours evenly divided by gender. No interviewer worked two days sequentially. E-mail invitations with survey URL (and reminders) sent out daily in equally sized batches proportional to final quotas. Interviewers available 24/7. Respondents could exit and re-enter surveys. Self-administered Web E-mail invitations with survey URL (and reminders) sent out daily in equally sized batches proportional to final quotas. Survey available 24/7. Pop-Up Exit Survey on Terminations Respondents could exit and re-enter surveys. Data collection results (Exhibit 12).

 Known technical barriers higher with e-interviewer because interviewers could ask respondents. Self-administered Web questionnaire required twice as many respondents going in. Technical problems Retention and completed response problems. Larger ineligible respondent count with self-administered survey increases sample requirements. Exhibit 12

Key benefits of e-interview approach

Significantly reduce min-termination (dropout) rates. From 44% dropout with 22-minute self-administered questionnaire to 31% with 37-minute e-interviewer questionnaire. Harris Interactive reports 44% dropout in a 22-minute questionnaire, and 75% with a 37-minute questionnaire. Modalis reports rates of 40% and 60% respectively. No rapport with interviewer (Exhibit 13).

Exhibit 13

e-interviews solve causes of mid-termination No rapport with interviewer Presence of open-end questions Respondent technical difficulties No opportunity to request question clarification or verification of sponsor. Current humanvoice system provides phone link when respondent requests a call. Future generation of software will include a voice IP connection. No option for respondent to change interview mode at will (Exhibit 14). Exhibit 14

Solving some knotty Internet challenges. Reducing mid-terms decreases sample panel fatigue and self-selection bias. Click to callback and instant messaging allow resolution of technical problems or interview mode change. Increase by 30% the number of respondents 55+ years who complete surveys. Increase open-end response length Important to B2B surveys. Consistent with previous research, findings indicate that e-interview open-ended response exceeds phone surveys.

 Fewer terminations on open-end questions when interviewer present. Average word length of open-end responses and follow-up probes is longer (Exhibit 15). Exhibit 15

Screen out ineligible and professional respondents. Frequency of professional respondents in the U.S. is on the rise Self-administered surveys have always suffered from ineligible respondents Self-selection rates are high in sample panels with high mid-termination rates Site http://www.money4surveys.com brings professionals to your site who want the money. Ineligible respondents caught by version. (Security required unique PIN and year of birth.) CATI: n=300 & 5 ineligibles e-Interviewer: n=117 & 2 ineligibles Self-Administered: n=122 & ?????????

It is inexpensive
For a survey featuring n=600 50 questions Listed, 80% incidence sample Re-invite after initial sample send Using SurveyGuardian 24/7 these are comparative costs: CATI: $11,356, including the high cost of callbacks. e-Interviewer: $8,414.

 Self-administered: $4,850. The fixed cost of programming can drive up the cost per response for small surveys. Costs will be different, probably higher, for B2B surveys.

Additional benefits

Open-ended questioning capability allows customer service options, such as immediate response to a customer complaint. Interviewer-collected data is cleaner in the database.
Return to the Table of Contents

Featured Research Overview:

Rethinking Marketing Research For the Digital Environment

Arvind Rangaswamy
Jonas H. Anchel Professor of Marketing and Research Director, eBusiness Research Center The Pennsylvania State University University Park, Pennsylvania

Key points

Traditional research migrating to online environment is analogous to the early years of television. TV started with radio formats exported to the new medium, until it developed its own unique formats. Pushing the online research envelope is the real golden goose promised by the medium. Myths about the online marketing environment are being challenged. Research finds: The more information buyers receive online, the less important the brand name as a cue to quality. Its total nonsense that online markets increase buyer price sensitivity. Buyers become more value sensitive. Price is considered a correlate to value, but the additional product information available online allows buyers to weigh other value factors as well. They dont necessarily buy the cheapest offering. About 60% of buyers use the Web for convenience; they do not use all the buying information available to them online. About 40% use Web information to optimize choices.

 There does not appear to be a difference in customer satisfaction online or offline, suggest studies by Marriott. But online customers tend to be more loyal because, it seems, its easier to remain loyal online if youre satisfied.

Types of marketing research on the Internet

Secondary data research Search engines; e.g., Google ordering responses according to respective sites network connection densities. Increasing use of bots will personalize online search. Content sites (e.g., myplant.com) Primary data research Surveys (e.g., e-mail, web site) Panels (e.g., focus groups, continuing panels, chat groups) Experiments; e.g. simulated test markets Observations. By observing what people do online, you can segment markets dynamically, and let customers self-select their segments. Primary data methods: Online Surveys New medium for traditional surveys Enhanced surveys Web-based surveys (e.g., www.insightexpress; www.harrisinteractive.com) e-mail surveys Web site evaluation surveys Online Focus Groups Focus group videoconferencing Focus group chat windows Continuing panels offering interesting B2B opportunities Longitudinal tracking studies Custom studies Web site statistics Standard Log files Enhanced Log files (e.g., Peapod) Typical Web survey method illustrated in Exhibit 16.

Exhibit 16

Web site evaluation surveys Content and Structure (Examples of items) Graphics Visual Attractiveness Selling Messages Links Chat rooms Registration Forms Audio Good sources for Web site evaluation: http://www.iconocast.com/ and http://www.emarketer.com/ Ease of navigation (e.g., Site search engine) Experience during visit How much did respondents enjoy their visit? Did visitors feel confused while using the site? Were visitors frustrated in any way with their experience? Did visitors find their visit exciting or boring? Did the site meet, exceed or fall short of visitors' expectations? What was visitors' overall level of satisfaction visiting the site? Likelihood of repeat visit

Polls apart?
Non-representative samples

Self-selection bias. Respondents are heavier users of computers, Internet and email than non-respondents Matching a sample to population on observable characteristics will not make it representative (e.g., propensity weighted scores wont work!) There are unobserved heterogeneities, but how can we observe what we cannot see? We must use methods to differentiate Web users from non-users. Why do some use the Web and others do not? Low response rates (e.g., banner clicks average around 0.30%) Problems of respondent authenticity Difficulties associated with incentives Difficulty in gauging response accuracy

www.modalis.com: Web-based surveys, e-mail surveys. Respondents recruited on the web. www.comscore.com: Generates company-specific panels and monitors their web behavior. www.greenfield.com: 2.2 million panelists who have volunteered to be members. Participates in drawing to win cash prizes. www.harrisinteractive.com: 7 million panelists worldwide.

Some online research suppliers

Potential online research benefits

Cheaper (Typically about one-third the cost of mail surveys) Faster than mail surveys Flexible (Multiple paths in surveys) Completion rates are higher Can reach elusive groups (e.g., CIOs) Richer content and context than mail and telephone surveys Interactive (useful for pre-test)

Improving Internet sample representativeness

Select samples from panels Pre-qualify and profile respondents Put banners/links to survey at popular web sites Offer incentives for participation. Even $2 can boost response. Focus on products of general usage (e.g., supplies) and segments that match the Internet population.

Big future for online focus groups, synchronous and asynchronous

Potential Benefits Reach difficult to recruit respondents (e.g., geographically dispersed, lowincidence, high currency) Broader geographic representation in focus group

Reduce travel costs Useful for discussing sensitive issues (requiring some anonymity) Quick turnaround (e.g., transcripts and keywords) Challenges Changes the dynamics of the communication: typing vs. voice. Handling emotive issues Some innovations Focus groups via videoconferencing Focus groups via chat windows

More innovation
Test market experiments Electronic shelf labels Computer-simulated test markets: a stable technology now that could have value for B2B, such as simulated trade shows. Promotional kiosks Customer tracking: e.g., point-of-sale linked to infrared sensors, now experimental.

Web site statistics

Few marketers use al the data Web sites generate How many users visit our site daily? Is that number growing? What paths do visitors take when they browse our web site? Which pages are the most popular? What kind of information is accessed on our server? How many pages are accessed in each directory? From what countries do users connect? What cities? What states? From what departments do users connect to the Intranet Server? Which is the most active day of the week? The most active hour? What browsers are used to access our web server? What operating systems? Which sites offer the best referrals to our pages? Web server logs Transfer Log (records each request to web server) Error Log (e.g., broken links, mid-process breaks) Referrer Log (e.g., source web addresses from which a user comes to a specific page) Agent Log (e.g., browser version of user) Transfer Log (common log format) Host name or IP address of the computer making the request User name of the user on the computer making the request (seldom used) User name on the local web site making the request (if the reader logs into a secure area of the web site) Time stamp - the date and time of the request Request - the text of the actual HTTP request, including the path and file names of the file requested

 Status code - the code for the resulting success or failure of the request Transfer volume - the number of bytes sent to the reader's browser as a result of the request. Sample log, Exhibit 17 Exhibit 17

Statistics from common log format Number of requests Number/percentage of successful/failed requests Number/percentage of cached requests Top pages or files (most requested documents) Number of page-transfers by day Top downloaded files by type (all files) Top submitted forms and scripts Bottom pages or files Top pages by directory Top directories accessed Average number of requests per week Average number of requests per day Total bytes transferred Average bytes transferred by day Average bytes transferred by hour of day Average number of hits on weekdays/weekends Most/least active day of the week (and number of hits) Most/least active day ever (and number of hits) Activity level by day of week/hour of day

Enhancements to the common log format Difficult to link information across log files Combined Log Format (Combines Transfer log, Referrer log, and Agent log). Difficult to identify unique visits Cookies (Stored in browser with expiration time) CGI Session ID (appended to URL) User-registration (Append ID information from any of these methods to log files) Statistics from enhanced log format Number of visits Average number of requests (and page views) per visit Average duration of a visit Sequence of user activities at the site Average number of visits per day or week Number of visits by hour of the day Visits from organizations (most active organizations) Visits by organization type (root domain) Visits from countries (most active countries) Top visit entry pages Top-page durations Top exit pages Average number of users on weekdays/weekends Visit level by day of week/hour of day Top U.S. geographic regions Percentage of visits from inside/outside the U.S. Top cities Top referring organizations Top referring URLs Top browsers Top user operating systems But statistics only tell you what people do, not why they do it. To derive marketing insights from Web site statistics, you need combined market data sources: e.g., Exhibit 18.

Exhibit 18

Source: Arvind Rangaswamy

Summary benefits of online research

Access to volumes of secondary research Potential for inexpensive, instantaneous, interactive, and global communication with customers More realistic marketing stimuli and decision contexts Ability to dynamically change marketing programs and measure consumer response Individual-level data on search and choice

Pushing the research envelope

Integrate marketing research with marketing planning and implementation Real-time research and analysis You can do a study analysis within an hour of gathering data with no loss of quality compared to current methods. Instant messaging technology can indicate who in your target audience is actually online. Link research to implementation (e.g., segmentation study) Make customers an integral part of the marketing planning process

 Establish data sharing learning communities for benchmarking. Link everyone in an iterative learning process, for example. Take advantage of the developing network of respondents. Do online experiments Develop methods and models for deriving insights from large data sets Create customized marketing research bots Real-time research and analysis: e.g. www.valueharvest.com indicates how data and analysis can be stored and run on different servers (Exhibit 19). Exhibit 19

Co-design products with customers. Texas Instruments designed a calculatorits most successful modelfor the educational market, with 30,000 educators responding to TIs invitations to comment. Factors going in Complex buy cycle Long adoption/legal issues Mixed price sensitivity Need support/curriculum Vendor credibility is key Active community of customers

 Insights gleaned The market owns the product! Therefore, the product owns the market. Reduces adoption times Web can concentrate influential audience Need responsive design team that can deliver Need a culture of experimentation Netscape stumbled into customer co-design with its version 3.0, because it had many engineers online. Establish data-sharing learning communities for benchmarking. Create virtual learning communities. Example: Companies pool their Web use information in the benchmarking survey of Penn States eBusiness Research Center (www.ebrc.psu.edu/benchmark) A model for sharing information, the survey software gives respondents an instantaneous report of all respondents collective input, as illustrated in Exhibit 20. Reports preserve respondent anonymity. Exhibit 20

Q&A
Q: How do you develop or find good B2B email lists?

A: Make your Web site the dominant information source in your market. Attract audiences by becoming the destination site.
Return to the Table of Contents