Are You playing to a Full House?
| Category: Surveys and Statistics | Date: 2001-03-22 |
Internet surveys show one big, happy audience, but here's what you really need to know about 'Net demographics before bringing the curtain up on your Web site.
British Statesman Benjamin Disraeli, 1st Earl of Beaconsfield, twice prime minister and purchaser of the Suez Canal is often remembered for pointing out, "There are three kinds of lies: lies, damned lies, and statistics." If the Prime Minister were around today he would certainly address his remarks toward the state of affairs of Internet surveys.
As a business person, you have to deal with bottom-line type questions. Questions like, is your intended audience on the Internet? Can you expect a reasonable ROI? How much should you invest in your Web site? To arrive at rational, logical answers you need to know who is on the Internet. How old are they? Do they have any money? Will they give some to you?
To resolve these mysteries, you turn to the myriad of surveys of the Internet and its denizens. Surveys from universities, surveys from Internet access providers, surveys from on-line content providers. Before digging into the results of these studies, you should take a moment to consider how they are conceived, gestated, and brought forth. This will help you make the best use of the results and give you a better appreciation for Mr. Disraeli's attitude toward statistics.
The Academic Approach
The majority of survey findings floating around the Internet and in the media these days are the type made possible by the very technology used to collect them. This sort of survey is created on the Web. "Wow, look!" says the enterprising graduate student. "We can create a form on a Web page and people can fill it out to tell us about who they are and what they like!" This is known as a self-fulfilling fallacy.
The Graphics, Visualization, & Usability Center at the College of Computing at Georgia Institute of Technology in Atlanta is mercifully known as the GVU. The GVU's 4th WWW User Survey (http://www.cc.gatech.edu/gvu/user_surveys/) was run from October 10th 1995 through November 10th 1995. Understand that "was run" means that they created a Web-based form for surfers to fill out on their way by.
As a result, we know something about the 23,000 people who were already on the Web, were able to find their way to the GVU Web sit, were willing to fill the survey, and clearly have too much time on their hands. Of course, we don't know what percent of the total population they represent, nor their motivation for answering a bunch of questions, but what the heck, it's only numbers. If you wish to rely on this methodology, then you can plan your Web presence for a 35 year old married man with an income of $63,000 who is on-line once a day for a total of 6 to 10 hours a week usually from work on a 14.4 modem.
But creating a business plan or avoiding the Internet because of this profile would be a mistake. And writing off this survey completely would mean missing some very interesting generalizations:
80% of respondents said they navigate primarily by Hotlist/Bookmark.
95% find out about WWW pages from other web pages.
51% paid for their Internet access themselves, 33% by work, and 25% by school.
62% use Windows and 21% use Macintosh.
The most common connection speed is 14 Kb/sec (34%) followed by 28 Kb/sec (27%).
Most people said they use the Internet mostly for downloading software updates or aquiring new software.
This was followed closely by people looking for reference information, with shopping getting the lowest rating.
The highest rated usage is "Replace Other Software" followed by reference information, and with shopping getting the lowest rating.
60% have been on the Internet less than 1 year, with 28% less than 6 months.
Figures That Mean Business
But if you're trying to get handle on a real picture of the average Internet user, you'll have to look elsewhere. Who's out there? How many are there? Can we somehow lump them into a nice, recognizable blob and call it a market? O'Reilly & Associates, Sebastopol (http://www.ora.com/), CA took a whack at it using Random Digit Dialing (RDD) and completing interviews with 29,901 individuals and only found an estimated 10 million Internet users. That's a third less than most estimators but then, they didn't count the on-line services, such as Prodigy, America Online, CompuServe & Microsoft Network.. They see the average user as a 36 year old male with an income of $63,000. Surprisingly close to the GVU study.
Matrix (http://www.matrix.net/index.html) Information and Directory Services, Austin, TX, "conducts ongoing investigations about the size, shape, and other characteristics of the Internet and other networks in the Matrix." Their latest survey results were released in December of 1994 indicating 13.5 million Web users and 27.5 million who do the electronic mail thing.
MIDS' next "How Big Is The Internet study" will be out soon, and will most likely interest Internet equipment makers and service providers rather than marketing folks. It focuses more on the architectural side of the coin than the buying side. They want to know how many people in your organization between the ages of 36 and 45 can FTP files out to the Internet. And nowhere else will you find a survey that offers 39 possible connection speeds to choose from.
The Nielsen Ratings
Not to be left out of the picture is the Nielsen company, those proctors of public opinion, who teamed up with CommerceNet (http://www.commerce.net/), that consortium of electronics corporations, to divine the truth and alleviate this ignorance plaguing our times.
We now take you to what would once have been a smoke-filled room, but today is filled with bottles of mineral water, Starbucks double cappuccinos, bran muffins and a team of well-intentioned questionnaire designers. They want to know how many people are on the World Wide Web. Why? Because their careers depend on the continued growth of the Internet. If their survey determines the Internet is populated by under-employed, over-sexed, penniless, male graduate students, they all have to find new jobs next week.
So, when it comes time to create just the right question asked in just the right way, the discussion goes something like this:
"Let's call random telephone numbers so we get a true cross-section."
"Right, then we'll ask them if they have a dial-up or a SLIP account."
"No, no, no. They may be connected at work or have America Online and not know what kind of connection they've got. In fact, they might use their parent's account or their friend's account."
"Right, so we ask, "Do you have access to the Internet or an on-line service at work and/or at home?"
"Or through a friend."
"Right. Or through a friend."
This is tantamount to asking if you have access to US Interstate 10. Do you have a car? Does your company have a car? Can you steal a car? Flag a taxi? Catch a Greyhound? Do you know somebody who can drive? Ah-ha! YOU have access. Therefore we can predict the flow of rush hour traffic and tell you if the people on that road are going to be prospective customers.
Stranger ways of calculating market size and value exist. For decades the Nielsen company has relied on television diaries filled out by a handful of supposedly normal people. The "Nielsen ratings" are derived from those viewing habits and projected onto the rest of the us. This is the standard way to determine TV market share and set air-time advertising fees. So, as a nation, we are already used to market evaluation based on people with too much time on their hands. Have you ever met a Nielsen family? They aren't plentiful.
It may be CommerceNet (who want to make a living showing everybody the technical vastness of the future) and Nielsen (which is accustomed to seeing the world through pin-hole eyeglasses) have offset each other's biases and produced the most reliable results to date. On the other hand, merely invoking the phrase "Nielsen Ratings" in a corporate board room is enough to lull unsuspecting management money mongers into loosening the purse strings. Therefore, their efforts are worth recounting.
They called thousands of statistically random people and managed to get 4,200 of them to answer 40 multiple-part questions (no mean feat to be sure). They were careful to categorize Internet users, on-line service users, and just plain old folks with better things to do.
According to the details at the CommerceNet Web site some of the key findings are:
17% (37 million) of total persons aged 16 and above in the US and Canada have access to the Internet.
11% (24 million) of total persons aged 16 and above in the US and Canada have used the Internet in the past three months.
Approximately 8% (18 million) of total persons aged 16 and above in the US and Canada have used the WWW in the past three months.
Internet users average 5 hours and 28 minutes per week on the Internet.
People in the United States and Canada are on the Internet the same amount of time that they are watching rented video tapes.
Males represent 66% of Internet users and account for 77% of Internet usage.
On average, WWW users are upscale (25% have income over $80K), professional (50% are professional or managerial), and educated (64% have at least college degrees).
Approximately 14% (2.5 million) of WWW users have purchased products or services over the Internet.
The Course of True Statistics Never Did Run Smooth
Now comes the kicker. Donna L. Hoffman (http://www2000.ogsm.vanderbilt.edu/), an associate professor at Vanderbilt University and a well known Web marketing academician, says these numbers are skewed. She says the survey tended to included more older, wealthier and higher educated people than would be statistically correct when compared to the 1990 U.S. census. And she should know. She was their principal academic advisor while the survey was being created. Now that she's seen the raw data, she's determined her advice wasn't heeded. Twenty-four million used the Internet in the past three months? More like ten million she says.
From her own Web site:
25% (9 + 15) of the Census respondents did not complete high school, vs. 11% of the CN/Nielsen Survey respondents.
Further, if we could eliminate 16 and 17 year olds from the CN/Nielsen sample to make it comparable to the Census data which is based on people 18 and older, we would expect the percentage of the CN/Nielsen sample with less than a high school education to be even *lower* than 11% and thus show even more bias. Thus, our analysis is conservative and is likely to understate the bias.
18% (12 + 6) of the Census respondents have a Bachelor's degree or higher, while 28% (17 + 3 + 8) of the CN/Nielsen respondents have a Bachelor's degree or higher.
The implication of this comparison, available from publicly posted Census data and demographic data made public by CommerceNet, is that the bias in educational attainment in the CommerceNet/Nielsen sample would be expected to inflate the estimates of number of users of the Internet reported by Nielsen.
Will we never know the truth about who is on the Internet? Are we doomed to spend the rest of our days wondering if we should spend more on the Web than on direct mail? Perhaps if we average the averages-- take a survey of the surveys. What might we find?
Compare and Conclude:
MIDS GVU O'Reilly CommerceNet
--------- -------- ----------- -------------
Date 10/94 5/95 10/95 11/95
Age N/A 35 36 N/A
Male N/A 82% 65% 66%
Female N/A 15.5% 35% 33%
Income N/A $69,000 $63,000 25% > $80,000
Married N/A 50% N/A N/A
hours/week N/A 9.6 N/A 5.5
Collage Grad N/A N/A N/A 64%
Total 27.5 million N/A 10 million 37 million
One could conclude that selling expensive and sophisticated guy toys might be the way to go. These men have money, are educated and like to surf the Web. One could assume a Web site designed for this modern model of modem-man would be a sure hit. One would be wrong. Way wrong.
What the Numbers Don't Tell You
All of these surveys are carefully created and hotly disputed. Statisticians have no end of fun poking holes in each other's methodologies. You may select the survey results that best support your approach, your plans and your budget request, but those numbers won't tell you that the Internet is very different from other forms of marketing.
The Internet is not about mass market. It is not a homogenized community of like-minded, like-income clones who will all flock to the product that fits their demographics to a tee. The Internet is not about homogeneity. It's not about conforming. It's not attractive to the same type of people who are all alike. It's attractive to lots of different people because people are different. Each one has a voice. Each one can find that which interests them in particular.
The real question is whether businesses on the Web are reaching the people they wish to reach. For the answer, we turn to a different kind of survey. In a recent study Jill Ellsworth of Oak Ridge Research (http://www.oak-ridge.com/) in San Marcos, TX, talked to almost 450 companies doing business on-line to see if they were actually making money on the Internet. 82% said they were. Are these companies counting demographic statistics? No. They're counting their own return. Are these companies selling to an Internet mass market? No. But almost 70% of them are reaching people they want to reach well enough to be satisfied with their ROI.
The Good News
This ROI took anywhere from a couple of weeks to a year and a half for sales-only sites and from 3 months to two years for marketing-only sites.
Want more good news? One of the most telling things about the various demographic surveys is how they compare to their previous incarnations. All of the surveys which have had more than one outing, point in the same direction; the Net is growing and the population is becoming more and more diverse.
But it would be a mistake to lump the populations of New York, Los Angeles and London together and calling it a single market. The same applies to the Internet. Web sites which are designed for the Internet as a mass market are designed with the broadcast medium in mind.
They are designed without the realization that the Web is a two-way communication device. It's not a TV, a radio or a magazine. It's much more like a telephone.
The surveys discussed above represent hard work done by scholars and savvy business people. But given this new type of medium, the value of the results is in question. As you pursue them, keep in mind the words of Stanley Marcus, chairman emertius, Nieman Marcus, "Consumers are statistics. Customers are people."
About the Author
Jim Sterne stays active as a public speaker and as a consultant, helping each client set Internet marketing goals and determine customer relationship strategies.
Please contact Jim Sterne at (805) 965-3184 or jsterne@targeting.com to find out how his talents might help you achieve your marketing objectives. Or visit his webpage at http://www.targeting.com
British Statesman Benjamin Disraeli, 1st Earl of Beaconsfield, twice prime minister and purchaser of the Suez Canal is often remembered for pointing out, "There are three kinds of lies: lies, damned lies, and statistics." If the Prime Minister were around today he would certainly address his remarks toward the state of affairs of Internet surveys.
As a business person, you have to deal with bottom-line type questions. Questions like, is your intended audience on the Internet? Can you expect a reasonable ROI? How much should you invest in your Web site? To arrive at rational, logical answers you need to know who is on the Internet. How old are they? Do they have any money? Will they give some to you?
To resolve these mysteries, you turn to the myriad of surveys of the Internet and its denizens. Surveys from universities, surveys from Internet access providers, surveys from on-line content providers. Before digging into the results of these studies, you should take a moment to consider how they are conceived, gestated, and brought forth. This will help you make the best use of the results and give you a better appreciation for Mr. Disraeli's attitude toward statistics.
The Academic Approach
The majority of survey findings floating around the Internet and in the media these days are the type made possible by the very technology used to collect them. This sort of survey is created on the Web. "Wow, look!" says the enterprising graduate student. "We can create a form on a Web page and people can fill it out to tell us about who they are and what they like!" This is known as a self-fulfilling fallacy.
The Graphics, Visualization, & Usability Center at the College of Computing at Georgia Institute of Technology in Atlanta is mercifully known as the GVU. The GVU's 4th WWW User Survey (http://www.cc.gatech.edu/gvu/user_surveys/) was run from October 10th 1995 through November 10th 1995. Understand that "was run" means that they created a Web-based form for surfers to fill out on their way by.
As a result, we know something about the 23,000 people who were already on the Web, were able to find their way to the GVU Web sit, were willing to fill the survey, and clearly have too much time on their hands. Of course, we don't know what percent of the total population they represent, nor their motivation for answering a bunch of questions, but what the heck, it's only numbers. If you wish to rely on this methodology, then you can plan your Web presence for a 35 year old married man with an income of $63,000 who is on-line once a day for a total of 6 to 10 hours a week usually from work on a 14.4 modem.
But creating a business plan or avoiding the Internet because of this profile would be a mistake. And writing off this survey completely would mean missing some very interesting generalizations:
80% of respondents said they navigate primarily by Hotlist/Bookmark.
95% find out about WWW pages from other web pages.
51% paid for their Internet access themselves, 33% by work, and 25% by school.
62% use Windows and 21% use Macintosh.
The most common connection speed is 14 Kb/sec (34%) followed by 28 Kb/sec (27%).
Most people said they use the Internet mostly for downloading software updates or aquiring new software.
This was followed closely by people looking for reference information, with shopping getting the lowest rating.
The highest rated usage is "Replace Other Software" followed by reference information, and with shopping getting the lowest rating.
60% have been on the Internet less than 1 year, with 28% less than 6 months.
Figures That Mean Business
But if you're trying to get handle on a real picture of the average Internet user, you'll have to look elsewhere. Who's out there? How many are there? Can we somehow lump them into a nice, recognizable blob and call it a market? O'Reilly & Associates, Sebastopol (http://www.ora.com/), CA took a whack at it using Random Digit Dialing (RDD) and completing interviews with 29,901 individuals and only found an estimated 10 million Internet users. That's a third less than most estimators but then, they didn't count the on-line services, such as Prodigy, America Online, CompuServe & Microsoft Network.. They see the average user as a 36 year old male with an income of $63,000. Surprisingly close to the GVU study.
Matrix (http://www.matrix.net/index.html) Information and Directory Services, Austin, TX, "conducts ongoing investigations about the size, shape, and other characteristics of the Internet and other networks in the Matrix." Their latest survey results were released in December of 1994 indicating 13.5 million Web users and 27.5 million who do the electronic mail thing.
MIDS' next "How Big Is The Internet study" will be out soon, and will most likely interest Internet equipment makers and service providers rather than marketing folks. It focuses more on the architectural side of the coin than the buying side. They want to know how many people in your organization between the ages of 36 and 45 can FTP files out to the Internet. And nowhere else will you find a survey that offers 39 possible connection speeds to choose from.
The Nielsen Ratings
Not to be left out of the picture is the Nielsen company, those proctors of public opinion, who teamed up with CommerceNet (http://www.commerce.net/), that consortium of electronics corporations, to divine the truth and alleviate this ignorance plaguing our times.
We now take you to what would once have been a smoke-filled room, but today is filled with bottles of mineral water, Starbucks double cappuccinos, bran muffins and a team of well-intentioned questionnaire designers. They want to know how many people are on the World Wide Web. Why? Because their careers depend on the continued growth of the Internet. If their survey determines the Internet is populated by under-employed, over-sexed, penniless, male graduate students, they all have to find new jobs next week.
So, when it comes time to create just the right question asked in just the right way, the discussion goes something like this:
"Let's call random telephone numbers so we get a true cross-section."
"Right, then we'll ask them if they have a dial-up or a SLIP account."
"No, no, no. They may be connected at work or have America Online and not know what kind of connection they've got. In fact, they might use their parent's account or their friend's account."
"Right, so we ask, "Do you have access to the Internet or an on-line service at work and/or at home?"
"Or through a friend."
"Right. Or through a friend."
This is tantamount to asking if you have access to US Interstate 10. Do you have a car? Does your company have a car? Can you steal a car? Flag a taxi? Catch a Greyhound? Do you know somebody who can drive? Ah-ha! YOU have access. Therefore we can predict the flow of rush hour traffic and tell you if the people on that road are going to be prospective customers.
Stranger ways of calculating market size and value exist. For decades the Nielsen company has relied on television diaries filled out by a handful of supposedly normal people. The "Nielsen ratings" are derived from those viewing habits and projected onto the rest of the us. This is the standard way to determine TV market share and set air-time advertising fees. So, as a nation, we are already used to market evaluation based on people with too much time on their hands. Have you ever met a Nielsen family? They aren't plentiful.
It may be CommerceNet (who want to make a living showing everybody the technical vastness of the future) and Nielsen (which is accustomed to seeing the world through pin-hole eyeglasses) have offset each other's biases and produced the most reliable results to date. On the other hand, merely invoking the phrase "Nielsen Ratings" in a corporate board room is enough to lull unsuspecting management money mongers into loosening the purse strings. Therefore, their efforts are worth recounting.
They called thousands of statistically random people and managed to get 4,200 of them to answer 40 multiple-part questions (no mean feat to be sure). They were careful to categorize Internet users, on-line service users, and just plain old folks with better things to do.
According to the details at the CommerceNet Web site some of the key findings are:
17% (37 million) of total persons aged 16 and above in the US and Canada have access to the Internet.
11% (24 million) of total persons aged 16 and above in the US and Canada have used the Internet in the past three months.
Approximately 8% (18 million) of total persons aged 16 and above in the US and Canada have used the WWW in the past three months.
Internet users average 5 hours and 28 minutes per week on the Internet.
People in the United States and Canada are on the Internet the same amount of time that they are watching rented video tapes.
Males represent 66% of Internet users and account for 77% of Internet usage.
On average, WWW users are upscale (25% have income over $80K), professional (50% are professional or managerial), and educated (64% have at least college degrees).
Approximately 14% (2.5 million) of WWW users have purchased products or services over the Internet.
The Course of True Statistics Never Did Run Smooth
Now comes the kicker. Donna L. Hoffman (http://www2000.ogsm.vanderbilt.edu/), an associate professor at Vanderbilt University and a well known Web marketing academician, says these numbers are skewed. She says the survey tended to included more older, wealthier and higher educated people than would be statistically correct when compared to the 1990 U.S. census. And she should know. She was their principal academic advisor while the survey was being created. Now that she's seen the raw data, she's determined her advice wasn't heeded. Twenty-four million used the Internet in the past three months? More like ten million she says.
From her own Web site:
25% (9 + 15) of the Census respondents did not complete high school, vs. 11% of the CN/Nielsen Survey respondents.
Further, if we could eliminate 16 and 17 year olds from the CN/Nielsen sample to make it comparable to the Census data which is based on people 18 and older, we would expect the percentage of the CN/Nielsen sample with less than a high school education to be even *lower* than 11% and thus show even more bias. Thus, our analysis is conservative and is likely to understate the bias.
18% (12 + 6) of the Census respondents have a Bachelor's degree or higher, while 28% (17 + 3 + 8) of the CN/Nielsen respondents have a Bachelor's degree or higher.
The implication of this comparison, available from publicly posted Census data and demographic data made public by CommerceNet, is that the bias in educational attainment in the CommerceNet/Nielsen sample would be expected to inflate the estimates of number of users of the Internet reported by Nielsen.
Will we never know the truth about who is on the Internet? Are we doomed to spend the rest of our days wondering if we should spend more on the Web than on direct mail? Perhaps if we average the averages-- take a survey of the surveys. What might we find?
Compare and Conclude:
MIDS GVU O'Reilly CommerceNet
--------- -------- ----------- -------------
Date 10/94 5/95 10/95 11/95
Age N/A 35 36 N/A
Male N/A 82% 65% 66%
Female N/A 15.5% 35% 33%
Income N/A $69,000 $63,000 25% > $80,000
Married N/A 50% N/A N/A
hours/week N/A 9.6 N/A 5.5
Collage Grad N/A N/A N/A 64%
Total 27.5 million N/A 10 million 37 million
One could conclude that selling expensive and sophisticated guy toys might be the way to go. These men have money, are educated and like to surf the Web. One could assume a Web site designed for this modern model of modem-man would be a sure hit. One would be wrong. Way wrong.
What the Numbers Don't Tell You
All of these surveys are carefully created and hotly disputed. Statisticians have no end of fun poking holes in each other's methodologies. You may select the survey results that best support your approach, your plans and your budget request, but those numbers won't tell you that the Internet is very different from other forms of marketing.
The Internet is not about mass market. It is not a homogenized community of like-minded, like-income clones who will all flock to the product that fits their demographics to a tee. The Internet is not about homogeneity. It's not about conforming. It's not attractive to the same type of people who are all alike. It's attractive to lots of different people because people are different. Each one has a voice. Each one can find that which interests them in particular.
The real question is whether businesses on the Web are reaching the people they wish to reach. For the answer, we turn to a different kind of survey. In a recent study Jill Ellsworth of Oak Ridge Research (http://www.oak-ridge.com/) in San Marcos, TX, talked to almost 450 companies doing business on-line to see if they were actually making money on the Internet. 82% said they were. Are these companies counting demographic statistics? No. They're counting their own return. Are these companies selling to an Internet mass market? No. But almost 70% of them are reaching people they want to reach well enough to be satisfied with their ROI.
The Good News
This ROI took anywhere from a couple of weeks to a year and a half for sales-only sites and from 3 months to two years for marketing-only sites.
Want more good news? One of the most telling things about the various demographic surveys is how they compare to their previous incarnations. All of the surveys which have had more than one outing, point in the same direction; the Net is growing and the population is becoming more and more diverse.
But it would be a mistake to lump the populations of New York, Los Angeles and London together and calling it a single market. The same applies to the Internet. Web sites which are designed for the Internet as a mass market are designed with the broadcast medium in mind.
They are designed without the realization that the Web is a two-way communication device. It's not a TV, a radio or a magazine. It's much more like a telephone.
The surveys discussed above represent hard work done by scholars and savvy business people. But given this new type of medium, the value of the results is in question. As you pursue them, keep in mind the words of Stanley Marcus, chairman emertius, Nieman Marcus, "Consumers are statistics. Customers are people."
About the Author
Jim Sterne stays active as a public speaker and as a consultant, helping each client set Internet marketing goals and determine customer relationship strategies.
Please contact Jim Sterne at (805) 965-3184 or jsterne@targeting.com to find out how his talents might help you achieve your marketing objectives. Or visit his webpage at http://www.targeting.com
Copyright © 2005-2006 Powered by Custom PHP Programming