Survivorship bias | Berkson bias | Why big data and experience can lie to you?

Survivorship bias | Berkson bias | Why big data and experience can lie to you?

Comments

Post a Comment

Popular posts from this blog

Statistics and probability

Image
Imagine you have an apple orchard farm, before harvest you want to know the taste of these apples.  But how? Take a bite of each one of them? Absolutely not. I guess you would randomly pick several apples and judge the overall sweetness based on the taste of those.  Wait a minute, why these apples can represent the whole orchard? Because you are using the statistics method. Actually, we just go through a famous statistical investigation (problem-solving) cycle: Problem, Plan, Data, Analysis, Conclusions, and Communication (PPDAC). According to Wikipedia: Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data .  The well planned statistical process allows us to have a peep of the overall picture from the sample.  Such a useful tool!  Even better, it can help us foresee the future. Have you ever wondered where sports betting companies’ offering odd coming forms? Do you know they always win? Guess...
Read more

What is Randomness

Image
Randomness is one of the most important pillars to support the statistic and probability temple.  Randomness is unpredictable. In the common parlance, randomness is the apparent lack of pattern or predictability in events. Please remember: Randomness is unpredictable but it not equal to uncertainty. Let’s say team A has a 60 percent chance to win the game, that percentage is a certain number although you can’t tell whether they will win or lose next time. We can consider randomness is that we know all the possibilities, the only thing we don’t know is which one will happen next. Flipping a coin, you don’t know whether it will show head or tail, but it must be one of them. Shuffle play music, although you can’t predict which song will play next, it must be one on your playlist.  On the other hand, uncertainty is we have no idea what can happen next. For example the Black swan theory - something that comes as a total surprise. Bay the way all swans in New Zealand are black, he...
Read more

Quantify probability

Image
We finally come to the last foundation of statistics and probability. Quantify probability. If I said, my favorite team more likely to win the next game you probably feel that is a vague  statement. I can do it better: my favorite team has an 80% chance to win the next game. How to come to that percentage? Well, you may think it must be very complex and even unachievable. Actually, there are only three principles when quantifying the probability. Permutation and combination method: let us see an example. What is the probability of a rolling dice stop showing 6? There are 6 positions the dice may end up: 1 2 3 4 5 6. 6 is one of them. So that the chance of getting 6 is 1to 6. The key to quantifying probability using the permutation and combination method is finding all possible outcomes. we can use permutation and combination to deal with a single random event but what if there are multi random events? Sum method: when calculating the chance of a happens or b happens we can sum each...
Read more