Regression to the Mean

By Juan Carlos

Definition

An improbable outcome will most likely be followed by an outcome that is nearer to the average.

In a regularly distributed system, the more one observes an outcome, the closer outcomes approach the mean and cluster around the average.

Why Use It

When thereā€™s a remarkable result, folks often unintentionally assign meaning to random chance when future results most likely return to the average. The phenomenon occurs in business, weather, sports, and many other areas.

For example, a gambler loses money for weeks, then suddenly goes on a winning streak. In another scenario, your friend comes down with a cold, they take an herbal remedy and immediately feel better.

Confusing a causal event and a statistically probable one is easy.

When to Use It

Any activity where chance plays a role, such as in sports, is where folks often tell themselves stories about likely results that ultimately bear little weight on outcomes.

Commentators in sports spin narratives about how and why an athlete will perform better or worse. Say a basketball player made 50 points in one game for the first time. A commentator might express how this will make the player nervous and cause them to play poorly in the next game. As predicted, the basketball player does worse in the next game. Still, itā€™s less to do with the story and more with statisticsā€”the likelihood of being closer to average after playing an exceptional game is high.

Itā€™s essential to keep the principle in mind when establishing causality between two things. Imperfect correlation is found in scenarios where the best consistently get worse, and the worst get better with time.

It can be hard to know if itā€™s regression to the mean, and when folks can, they will rely on a control group to measure against the one exhibiting uncommon performance. The goal is to know whether the original test with spectacular results was a fluke.

How to Use It

Correlation, the strength of the association linking two variables, is the gateway for comprehending regression to the mean. The correlation coefficient between two variables varies between -1 and 1.

  • A negative one indicates a perfect negative correlation. Negatively correlated variables run in opposite directions. For example, if a train increases speed, the length of time to get to the final point decreases.
  • Zero exposes no linear relationship between the variables. Uncorrelated variables do not hold a linear relationship. A person watches an excessive amount of television, but that has no bearing on its size.
  • One designates a perfect positive correlation. Positively correlated variables run in the same direction. The more hours you spend in direct sunlight, the more severe your sunburn.

A perfect positive or negative correlation is unlikely. Most relationships between variables are somewhere in between -1 and 1.

Where correlation is imperfect, there will be a regression to the mean. The lower the correlation, the higher the regression and vice versa. For example, the longer someone invests, the more compound interest that person will earn. There is a positive correlation between investing and compound interest. Some folks who invest might yield more interest than others, but those unlikely results average out against others who earned less and invested the same amount over time.

How to Misuse It

Sometimes incredible results that deviate from the norm are not random or chance. The point is to know and notice when that is the case.

Next Step

One-time outliers exist statistically more often than we think, and tracking more instances of a given scenario demonstrates those abnormal results.

Turning to the principle first when looking for causes can help you make choices based on invalid reasoning. A more rigorous when considering performance and luck will help you make better decisions.

So many inexplicable phenomena can be attributed to the mathematics of regression to the mean.

Where it Came From

In 1886, Sir Francis Galton was conducting a study on childrenā€™s heights when he discovered the principle coining the term ā€œregression to mediocrity.ā€ Over time the mental model has been renamed ā€œregression to the meanā€ or ā€œreversion to the mean.ā€