Using and Abusing Comparisons
When we talk about social problems and how to solve them, we often make comparisons between cities, counties, political parties, religions, and so on. Politicians and talking heads do the same. These two obvious observations make a person wonder how to know if a comparison is fair or not. Well, how do you know? If we’re comparing two economic systems or two sets of gun laws, we should first decide what a fair comparison would look like. If we encounter a comparison in the media or in a political speech, we might want to know if the comparison is a fair one.
Choosing What to Compare with What
If we want to talk about gun control, comparing laws in the United States and laws in other countries, what countries do we choose? If we want to compare the quality of healthcare in other countries, to what countries should we compare the United States?
Opponents of gun control, supporters of gun control and virtually any person with a cause to promote can find a way to criticize a comparison. In gun control debates, the contrast between Japan and the United States comes up. Gun violence is quite rare in Japan, but gun ownership is illegal with a few exceptions. Japan allows shotguns with a single or double barrel. Getting a permit for one of those shotguns is not easy. Rifles are banned, but anyone with a license issued decades ago is allowed to own one. The United States has the Second Amendment, and by world standards, our gun laws are quite permissive. Gun violence is also much more common than in Japan. You are probably more likely to slip on ice and break your neck than you are to be shot. This doesn’t mean we should adopt Japanese gun laws. After all, Japanese culture is different. Laws that work there may not work here, assuming gun ownership could be banned in the United States, which is something that will never happen.
But, what country makes for a better comparison? That’s hard to say, and political bias makes a hard job effectively impossible. Do we compare ourselves to a wealthy nation with permissive gun laws, like Sweden, or with a poor nation like Pakistan? Do we choose a nation with a high rate of violent crime? A low rate? Why?
This is a common problem when anyone talks about social issues. What variables do we talk about? Is the rate of gun ownership more important than the poverty rate or the population density? How do we decide? And how many countries do we choose for our little comparison? If we choose only two nations, we have to remember an important point about social science research: One case doesn’t prove anything? A country with liberal gun laws and lower violent crime doesn’t necessarily tell us gun laws don’t work.
Sometimes that logic just gets overlooked. People make honest mistakes all of the time. Other times, people with a cause to promote, know exactly what they are doing, or else they just don’t care about their shaky logic at all.
How to Make Dishonest Comparisons
Cherrypick your numbers or your sample. Or, ignore different definitions of keywords. Those are the only three tricks you need to know. Let’s illustrate the three techniques with a dubious comparison of violence in Sweden and in the United States. In which country is rape a bigger problem, and how do we know?*
Ignore different definitions of the same concept. It can be tempting to assume ‘rape’ means the same thing in every country and so rape statistics mean the same thing. This is not the case. For example, in Sweden, I believe credible threats of rape and statutory rape count. Statutory rape is the crime of having sex with someone the law says is unable to consent. Typically, children and people with serious mental illnesses or intellectual disabilities aren’t able to give consent under the law. People who are legally drunk also can’t consent to have sex, at least in many countries.
What will happen if you compare the frequency of rape in the United States and Sweden if the definition of ‘rape’ is different? You can’t do it without lots of digging into the relevant crime statistics. In the United States, forcible rape is a distinct category from statutory rape. If you unknowingly compare the United States to other countries, where the definition of rape is broader, you will find that the United States looks like a safer place for women.
Here is what Wikipedia has on rape in Sweden:
Note the definition in the first paragraph.
Now, what about the United States? Note the definition in the first paragraph.
If you want to know how safe the United States is, in general, compared with other countries, you can compare rates of violent crime. Maybe. In truth, you have the same definition issue as with rape but for other crimes. There might be a way around this definition problem though. How about if we compare homicide rates? Homicides are not reported the same way everywhere, but murder is basically murder. Right?
It doesn’t really matter if homicides are recorded in exactly the same way everywhere. If you are an astute reader, you may have guessed why this is. Homicide is not a synonym for violence, is it? In fact, many serious crimes count as violent crimes.
Mind Your Comparisons
Whenever someone talks to you about a social problem, what causes is it or what we should do, they might try to compare things. Think about the comparison they choose. Does it make sense? Are the two things really comparable, or do people just WANT them to be comparable? Often, someone with an agenda will compare apples and oranges consciously, hoping you don’t know the difference between apples and oranges.
*Unfortunately, this comparison between rape in the United States and Sweden sometimes gets used by xenophobes and racists who want to argue that immigration from Black or majority Muslim nations leads to more violence.
