The Datasaurus Dozen

"If your data comes from a probability distribution that doesn't have a mean then calculating the mean of your data is basically meaningless (no pun intended)." I think this is actually the major takeaway for this sort of discussion. All statistical measures carry with them assumptions about the underlying distribution. When you blindly use one without verifying the underlying distribution, you are asking to be lied to by your statistics. It took me a while, but I've trained myself to actively ask the question "is this appropriate for the underlying distribution" whenever I see a "mean" or a "standard deviation". Spoiler: The answer is usually "no"! Central limit theorem notwithstanding, we encounter a lot of non-Gaussian and outright pathological distributions in the real world. Average is useful for more than just Gaussian, but it means different things in different distributions, most of them quite unlike what our Gaussian-trained intuition suggests. Standard deviation is really Gaussian-only, or at least, if you insist on using it, you ought to pair it with the skew, kurtosis, or other measures of how non-Gaussian your distribution is. Remembering that there's only one way to be Gaussian but an infinity of ways to be non-gaussian is helpful too. This can be a helpful video to visualize that: ▶ this video