Are You Guessing - Or Testing?
| Category: Surveys and Statistics | Date: 2003-03-17 |
Having trouble deciding which slogan to use on your Web page? Or what to call your newsletter? Or what to charge?
Well, youre on the Web now, and that means you can do quick, easy, and inexpensive testing to get answers. Ive done it a number of times and its been a useful experience each time. Here are three examples: Change and Measure; A Banner Example; and Online Questionnaire.
Change and Measure
A key objective for one of my websites is to get visitors to subscribe to my newsletter, Abbotts Communication Letter. When I started working to improve the ratio, the conversion rate was about 9%; that is, about one in 11 visitors subscribed.
In a bid to improve that ratio, I added a strong Because section at the top of the index page, explicitly listing the benefits of subscribing. Again, I measured responses, getting several hundred of them before drawing any conclusions. And what did I find? Adding the Because section improved the ratio, from 9% to 15%; now one in 6.6 visitors were subscribing.
To do this kind of testing, vary just one detail at a time, measure the results (with between 100 and 400 responses), and if the change worked, make it permanent and start over to test another change.
A Banner Example
Heres another challenge that seemed natural for testing: Which of two slogans should I use on the front cover of my book, A Managers Guide to Newsletters: Communicating for Results?
Solution: I created banner ads for each slogan, and tested them with the help of Mike Scanlin of Four Corners Effective Banners. Each banner got 3,000 impressions and when the test was done I had my answers.
To test with banners, start with a control banner and change just one thing on a second banner. Then review the click-through statistics to find out which works best. Of course, you can then take the best one and start testing it.
Online Questionnaire
Heres another type of testing: an online pricing questionnaire. I wanted to sell more copies of the electronic version of my book and wondered what the optimal price might be.
At the same time, Internet entrepreneur Ken Evoy developed a way to test prices using the power of the Internet. His online surveys use the Internet to gather, analyze and deliver the data at <http://myps.sitesell.com/acl.html>.
In this test, visitors to my website were given information about the book (but not the price) and asked to complete a quick, six-question survey. In exchange, they got a free report and a big discount if they bought the book.
When I had received enough responses to the survey, I was able to see clearly the optimum price and the effects that different prices would likely have on sales and profits.
How to Test
If you try to test a lot of concepts at once it gets complicated. Believe me, Ive tried. So change just one detail at a time,measure the results, compare with the original, keep or reject the change, and try another.
How many responses should you get? Try for 400 responses if you want to assess the opinions of more than 10,000 people, even millions of people. Does 400 seem too few to measure what millions of people might do? Well, assuming your test is reasonably well designed and youre getting a fairly representative sample of the population, 400 provides a commonly accepted level of accuracy. For example, political pollsters can accurately forecast the voting intentions of literally millions of people after talking to just 400 of them.
Can you get by with fewer than 400 responses? Sure. You wont have as much confidence in the results you get, and thats okay if the stakes arent high or if youre surveying a smaller population.
The important thing is that you test, not guess, because testing is the foundation of a successful initiative. And, using the power of the Internet makes it quicker, easier, and cheaper than ever before to test. Thats a deal you dont want to pass up.
About the author.
Robert F. Abbott, the author of A Managers Guide to Newsletters: Communicating for Results, writes and publishes a free online newsletter with communication ideas for leaders and managers abbottletter.com
robert@abbottletter.com
http://www.abbottletter.com
Well, youre on the Web now, and that means you can do quick, easy, and inexpensive testing to get answers. Ive done it a number of times and its been a useful experience each time. Here are three examples: Change and Measure; A Banner Example; and Online Questionnaire.
Change and Measure
A key objective for one of my websites is to get visitors to subscribe to my newsletter, Abbotts Communication Letter. When I started working to improve the ratio, the conversion rate was about 9%; that is, about one in 11 visitors subscribed.
In a bid to improve that ratio, I added a strong Because section at the top of the index page, explicitly listing the benefits of subscribing. Again, I measured responses, getting several hundred of them before drawing any conclusions. And what did I find? Adding the Because section improved the ratio, from 9% to 15%; now one in 6.6 visitors were subscribing.
To do this kind of testing, vary just one detail at a time, measure the results (with between 100 and 400 responses), and if the change worked, make it permanent and start over to test another change.
A Banner Example
Heres another challenge that seemed natural for testing: Which of two slogans should I use on the front cover of my book, A Managers Guide to Newsletters: Communicating for Results?
Solution: I created banner ads for each slogan, and tested them with the help of Mike Scanlin of Four Corners Effective Banners. Each banner got 3,000 impressions and when the test was done I had my answers.
To test with banners, start with a control banner and change just one thing on a second banner. Then review the click-through statistics to find out which works best. Of course, you can then take the best one and start testing it.
Online Questionnaire
Heres another type of testing: an online pricing questionnaire. I wanted to sell more copies of the electronic version of my book and wondered what the optimal price might be.
At the same time, Internet entrepreneur Ken Evoy developed a way to test prices using the power of the Internet. His online surveys use the Internet to gather, analyze and deliver the data at <http://myps.sitesell.com/acl.html>.
In this test, visitors to my website were given information about the book (but not the price) and asked to complete a quick, six-question survey. In exchange, they got a free report and a big discount if they bought the book.
When I had received enough responses to the survey, I was able to see clearly the optimum price and the effects that different prices would likely have on sales and profits.
How to Test
If you try to test a lot of concepts at once it gets complicated. Believe me, Ive tried. So change just one detail at a time,measure the results, compare with the original, keep or reject the change, and try another.
How many responses should you get? Try for 400 responses if you want to assess the opinions of more than 10,000 people, even millions of people. Does 400 seem too few to measure what millions of people might do? Well, assuming your test is reasonably well designed and youre getting a fairly representative sample of the population, 400 provides a commonly accepted level of accuracy. For example, political pollsters can accurately forecast the voting intentions of literally millions of people after talking to just 400 of them.
Can you get by with fewer than 400 responses? Sure. You wont have as much confidence in the results you get, and thats okay if the stakes arent high or if youre surveying a smaller population.
The important thing is that you test, not guess, because testing is the foundation of a successful initiative. And, using the power of the Internet makes it quicker, easier, and cheaper than ever before to test. Thats a deal you dont want to pass up.
About the author.
Robert F. Abbott, the author of A Managers Guide to Newsletters: Communicating for Results, writes and publishes a free online newsletter with communication ideas for leaders and managers abbottletter.com
robert@abbottletter.com
http://www.abbottletter.com
Copyright © 2005-2006 Powered by Custom PHP Programming