Hi there! This is the last of a 3 part post about Maths, Stats and Think Traps.  I hope you have enjoyed the trip so far! (make sure to checkout part 1 and part 2 if you have not yet).

This last post is about the Simpson’s Paradox which might destroy your faith in statistics forever so please thread carefully. I can’t remember exactly where I first found out about this problem but James Grimes from Numberphile does a great job in this video. The most common example about the Simpson’s Paradox has to do with college admissions but his example is way better so I will use it instead.

Let’s say that we found two different treatments for the disease that was ravishing the world on the last post. We decide to split up and go around the world finding sick people and treating them. In the first day I find 65 people who are sick and, with treatment 1, I manage to save 45 of them. You on the other hand find 35 people and save 25. It seems like treatment 2 (your treatment) is saving a bigger percentage of people:

Second day,  we get two new groups of people and we treat them again. Once again treatment 2 saves a bigger percentage of people:

At this point you might be thinking, everything looks OK, treatment 2 seems to be the way to go, take that Tiago! Well, hold your horses! Let’s add the numbers together and see what happens:

Turns out treatment 1 actually saved more people overall. In fact, maybe treatment 1 was just a sugar pill in which case you might have been feeding them poison the whole time. How can this be? Well, the answer has to do with the fact that in each day the number of people in each group is different so the groups are actually not comparable.

You might already be thinking ‘Fine, from now on, I will just consider the aggregate result whenever I am in a situation like this because this is clearly the correct way of looking at this thing’. This is where things start to become scary. Let’s take a look at another example:

Let us assume that women are more resistant to this disease (for some unknown reason). As it turns out, they also have more estrogen but, as you can see in the plot below, there is a clear correlation between estrogen and survival rate for both groups. As one goes up the other goes down:

However, if you ignore the fact that these are two different populations then you would be tempted to trace a different line of best fit and come up with a completely different answer (and you would be wrong).

In this case it is clear that, unlike the first example, the correct way of looking at the data is the one where each group is looked at separately. So… how do you know when to aggregate your data and when not to?

I hope you enjoyed these series of posts! If you want me to do more just let me know at any social media I posted this in.

See you later!