Next Question
RSS
You do not have an adequate sample for a P-value test (in the data provided). So I am guessing you want to run a P-value test against the actual data. If you actually have the 60,000 samples in a spreadsheet somewhere, then its a pretty simple thing to run a regression against them and get the revenue/1000 (with associated probabilities). You will need the Excel Analysis Toolpak add-in, and with that add-in select the "Regression" report. Just define the regression variables and create the regression. It's that simple.
Source(s):
http://www.jeremymiles.co.uk/regressionbook/extras/appendix2/excel/
Permalink | Report
http://yudkowsky.net/rational/bayes
Permalink | Report
While I was able to complete the calculations based on one set of data, I could never verify / repeat the result or calculation to any degree of validity.
1)
Today, I know a bit more about traffic than I did in the past. It is fickle and somewhat unpredictable. While you can easily measure and predict things like CPC, CTR, etc, with enough historical data, it is inaccurate to compare two different campaigns in with textbook statistics.
2)
History and demographics are the key. You need to sample data over common lengths of time with highly targeted traffic to form predictable results.
A long lineage of conversion results, ctr, etc will give you the data that you need to work with. 60,000 impressions isn't a long lineage. Also, if the traffic (the source) of what you're trying to measure isn't consistent then your results will flail around unpredictably.
3)
Right now, you're calculating simple ROI. It's not a bad way to calculate how effective a campaign is, but for accurate results, you need to consider the other variables associated with the campaigns.
Simple variables include: ad rotation numbers and verticals, % of target demographic reach, residual conversions vs. direct conversion, etc.
4)
Here is a handy calculator. It's even handier when the logic is coded directly into campaign analytics so that real time data can be observed:
http://www.toad.net/~jkaplan2/martMean.htm
5)
The easy answer is of course, "if the ROI is better with campaign 1, stick with that".
The only time that I ever measure things to this level is when I need to make a long term commitment or am pre-paying for results.
The rest of the time, I've found that I make more money when I redistribute traffic to a up-and-coming source of revenue. Remaining fluid and "rolling with the punches" is key to maximizing the effectiveness of each campaign.
Source(s):
I have been professionally involved in high level traffic and campaign management.
Permalink | Report
Answered Question
M$10
February 27, 2009 08:13 PM
Comparing Revenue per 1000 impressions, statistically -- advice
I'm trying to compare the results of two different approaches to generating revenue on online adverising - they generate different amounts of Revenue per 1000 Impressions. I believe Bayesian Reverse Posterior Probability Test is the way to go here but I cannot find a calculator for this.
i.e., approach #1 60,000 impressions, 1000 actions, $6,000 revenue and Rev/1000 is $100
approach #2 60,000 impressions, 3000 actions, $4,000 revenue and Rev/1000 is $66.67
I'd like to calculate P-value and ideally some confidence intervals that approach #1 is better than approach #2, say with 90% confidence it is 5%-10% better on Rev/1000
Thoughts?
i.e., approach #1 60,000 impressions, 1000 actions, $6,000 revenue and Rev/1000 is $100
approach #2 60,000 impressions, 3000 actions, $4,000 revenue and Rev/1000 is $66.67
I'd like to calculate P-value and ideally some confidence intervals that approach #1 is better than approach #2, say with 90% confidence it is 5%-10% better on Rev/1000
Thoughts?
Interesting Question?
Yes (0)
No (0)
- In Science & Mathematics |
- |
- Report |
-
Share
RSS
Best Answer Chosen by Asker
| February 27, 2009 09:48 PM |
Source(s):
http://www.jeremymiles.co.uk/regressionbook/extras/appendix2/excel/
| Asker's Rating: |
Permalink | Report
Reply
Other Answers (2)
February 27, 2009 08:57 PM
I like Bayesian too. http://yudkowsky.net/rational/bayes
Permalink | Report
Reply
March 03, 2009 01:46 PM
Early on in my experience measuring campaigns and evaluating their effectiveness, I tried to calculate p-values, probability, etc. While I was able to complete the calculations based on one set of data, I could never verify / repeat the result or calculation to any degree of validity.
1)
Today, I know a bit more about traffic than I did in the past. It is fickle and somewhat unpredictable. While you can easily measure and predict things like CPC, CTR, etc, with enough historical data, it is inaccurate to compare two different campaigns in with textbook statistics.
2)
History and demographics are the key. You need to sample data over common lengths of time with highly targeted traffic to form predictable results.
A long lineage of conversion results, ctr, etc will give you the data that you need to work with. 60,000 impressions isn't a long lineage. Also, if the traffic (the source) of what you're trying to measure isn't consistent then your results will flail around unpredictably.
3)
Right now, you're calculating simple ROI. It's not a bad way to calculate how effective a campaign is, but for accurate results, you need to consider the other variables associated with the campaigns.
Simple variables include: ad rotation numbers and verticals, % of target demographic reach, residual conversions vs. direct conversion, etc.
4)
Here is a handy calculator. It's even handier when the logic is coded directly into campaign analytics so that real time data can be observed:
http://www.toad.net/~jkaplan2/martMean.htm
5)
The easy answer is of course, "if the ROI is better with campaign 1, stick with that".
The only time that I ever measure things to this level is when I need to make a long term commitment or am pre-paying for results.
The rest of the time, I've found that I make more money when I redistribute traffic to a up-and-coming source of revenue. Remaining fluid and "rolling with the punches" is key to maximizing the effectiveness of each campaign.
Source(s):
I have been professionally involved in high level traffic and campaign management.
Permalink | Report
Reply
March 03, 2009 07:21 PM
Rob,
That's good stuff. You clearly have experience in this field. I do as well and I wanted to share my thoughts - largely conforming with yours
With your #1 point, if these two approaches have been running side-by-side in random rotation with sticky cookies at same percent for equal amounts of time, I think you can compare the two (in fact, if you are trying to maximize yield, you have to.)
Your #2 points out that the if the sample traffic is heterogeous over time, you will get wildly fluctuating results and I have seen that as well.
#3 I agree and what to do with that depends on your circumstance and trackability
#4 - thanks for the calculator too.
#5 - makes sense but wouldn't some statistical test help you try to decipher signal from noise when trying to figure out if you have enough info to make an informed decision about redistributing traffic?
Report
That's good stuff. You clearly have experience in this field. I do as well and I wanted to share my thoughts - largely conforming with yours
With your #1 point, if these two approaches have been running side-by-side in random rotation with sticky cookies at same percent for equal amounts of time, I think you can compare the two (in fact, if you are trying to maximize yield, you have to.)
Your #2 points out that the if the sample traffic is heterogeous over time, you will get wildly fluctuating results and I have seen that as well.
#3 I agree and what to do with that depends on your circumstance and trackability
#4 - thanks for the calculator too.
#5 - makes sense but wouldn't some statistical test help you try to decipher signal from noise when trying to figure out if you have enough info to make an informed decision about redistributing traffic?
March 03, 2009 07:49 PM
I'm disappointed that I didn't win this answer. Oh well.
#5) "wouldn't some statistical test help you try to decipher signal from noise".
Yes, as mentioned, I employ and encourage advanced statistical analysis when large or lengthy decisions need to be made. However, for the most part, basic analytic and ROI tracking are sufficient.
It is much more productive to spend time analyzing your creatives, placement, demographic reach, etc.
I could go on-and-on about this sort of stuff all day, but I'll leave it at that. I hope that my answer was at least helpful!
Report
#5) "wouldn't some statistical test help you try to decipher signal from noise".
Yes, as mentioned, I employ and encourage advanced statistical analysis when large or lengthy decisions need to be made. However, for the most part, basic analytic and ROI tracking are sufficient.
It is much more productive to spend time analyzing your creatives, placement, demographic reach, etc.
I could go on-and-on about this sort of stuff all day, but I'll leave it at that. I hope that my answer was at least helpful!
Answer this Question
Related Questions
Ask a Question
Buy Mahalo Dollars with Credit Card or PayPal
Top Members
Most Popular Tags
Categories
- Anonymous
-
Arts & Design
-
Beauty & Style
-
Books & Authors
-
Business
-
Cars & Transportation
-
Consumer Electronics
- Coupons Deals
-
Education
-
Entertainment
-
Environment
-
Fitness
-
Food & Drink
- From Email
- From Iphone
- From Twitter
-
Health
-
History
-
Hobbies
-
Home & Garden
-
How Tos
-
Humor
-
Jobs
-
Legal
-
Local
-
Love & Relationships
-
Mahalo Answers Community
-
Money
-
Music
-
News
-
NSFW
-
Parenting
-
Pets
-
Science & Mathematics
-
Services
-
Shopping
-
Social Science
-
Society & Culture
-
Sports
-
Technology & Internet
-
Travel
-
Video Games
Welcome New Members
- emerson, December 05, 2009 11:11 PM
- emersonbermudez, December 05, 2009 11:09 PM
- arjun47, December 05, 2009 10:55 PM
- tpittman3161, December 05, 2009 10:49 PM
- s1artist, December 05, 2009 10:44 PM
Mahalo Dollars are the currency of Mahalo Answers.
Each Mahalo Dollar costs $1.
Once you earn more than 40 Mahalo Dollars, you can request to be paid via PayPal. Each Mahalo Dollar is currently worth $0.75 when paid out via PayPal. Learn More
The distributions of revenue per visitor are clearly NOT normally distributed. Its some kind of binomial regresssion or something.
Can we compare those in excel or using a calculator?