I hate to post another xkcd, but this one is just too good.
Of course, this works with any group of "others," and does not apply purely to math. "Mexicans are lazy," or "Jews are money-grubbing," or "Muslims are terrorists" or even, let's say, "Christians are dumb," fits into this same error of categorization. I know that some will read this and pull out bullshit Bell Curve-like studies that purport to use bogus statistics and, you know, those number-thingies to prove their points, to which I only have to say two words: confirmation bias.
(Placed under "politics" because it's such a common political trope that it deserves inclusion there.)