Daily Fantasy Tournament Variance

by bripc23 Last Updated 2014-09-11 02:21:54

This article will explore the concept of variance and sample size in DFS tournaments. Variance in tournaments is driven by two key factors:

1. The small percentage of the player pool that gets paid, usually somewhere between 10-20%

2. The top-heavy nature of the payout structure

Given this, it is worth asking the question of when results become stable enough to make inferences about a player’s skill. In other words, how many games do we have to play until we have a good idea of our real ability?

DraftKings High Heat Example

I will use last Friday’s High Heat tournament on DraftKings as an example. To refresh your memory, here is a summary of that tournament:

• $109 entry fee
• 250 participants
• 50 positions paid according to the following structure:

Hypothetical Player

I created a hypothetical player to play in this tournament. This person cashes 26% of the time. Why did I choose this amount? Because it implies an ROI of 19%. Analyzing my results and talking to people in the industry I believe this number to be at the high end of what a very good player can achieve in the long-run. Whether or not you agree with this number is not that important – what is important is the variation around the number, which I will demonstrate.

Methodology

I simulate my hypothetical player in the DK tournament as follows:

• Generate a random number from 0 to 1
• If this number is below .26, consider it a “cash,” if above .26, a loss
• If a cash, generate another random number from 1 to 50 for finishing position and corresponding prize
• Repeat this process one million times

I did this and got the following graph. The red line is cumulative ROI across all trials up to that point, and the green line is the 19% long-term ROI that I referenced earlier.

So now the question is when do things stabilize? This is a bit dependent on personal opinion. On the one hand, you can say at around 1,300 trials things start to fall along the green line. On the other hand, ROI dips all the way down to 9% at the 3,000 trial mark, even though true ability of 19% has not changed, and does not seem to come back until about 6,000 trials.

[It is worth noting that this assumes all trials are completely uncorrelated with each other. If you are entering multiple lineups with similar core players across many lineups, than these volatility figures would be higher for you.]

Risk of No Profits

Although it is technically not defined this way, when people talk about variance, they typically are referring to a risk of losing money. Therefore, I approached the analysis from another angle. I broke my sample into intervals and analyzed profitability (or lack thereof) over these smaller samples. You can then ask yourself the following question: given a certain number of tournaments, what is the percent change of losing money? For example, for a range of 100 consecutive tournaments, the picture looks like this (ROI on the x-axis):

In this case, my player ends up losing money over the course of 100 tournaments 48% of the time. Clearly this is too small of a sample size, so let’s look at 1,000 tournaments.

Here the picture looks much better and more normally distributed. In this case, a 19% ROI player will be a loser over a 1,000 tourney sample 17% of the time. In fact, it took until 2,600 trials until I reached a threshold when there was less than a 5% chance of losing money. What does 2,600 trials feel like? Well, if you were to put a full 10 entries into a DK tourney every night for the entire baseball season, you would be at roughly 1,800 combined entries. In other words, if you are a true-talent 19% ROI DFS player, and play 10 lineups at DK every night for 6 months, you still have a very reasonable chance (more than 5%) of not making any money.

Conclusions

So what do we learn from this?

• Variance is tournaments is very high, even higher than I originally had thought
• Variance can be reduced by entering multiple lineups, but only if these lineups are at least partially uncorrelated with one another and by playing less top-heavy tournaments and tournaments that pay a larger percent of the player pool
• Even if you are a profitable player, it still takes over 1,300 tournaments to get a true sense of your ability, and even then it is only a rough estimate. 6,000 is better.

Alright, congrats to those of you that read that far. Be on the lookout for Part 2 where I analyze variance in cash games.