The other day I posted a statistic to my facebook profile – that 70% of adult males carry edged tools, such as knives, on a day to day basis in the U.S. A couple of people called me on the statistic; I have no problem with that. I do have statistics and research methods training from graduate school, so I think I can shed some light here. Let’s look at the use and misuse of statistics in self defense.
First the 70% statistic may or may not be accurate. I got it from a trusted source, and when I asked him for his source – he had legitimately lost his notes – but pointed me in to a couple of places where it may have come from. I’m still looking. That number includes blades carried for work purposes, multi-tools, and pocket knives. It seems like a reasonable number in my experience. A recent Gallup poll said that 11% of Americans responded that they carried knives for self defense, and that number was higher among men than women.
I was using the statistic the way statistics are often used – to convince people to take action. That action was to get some training in how to handle edged weapon assault. Attacks with edged weapons are fairly frequent in the U.S. and that includes the use of swords and machetes. Knives are very common. Let’s be clear that I am not selling training at this point, and am not using that statistic to scare people for my own personal gain. The real question about the value of the number is this – would following the interpretation of the statistic and getting some training be a good thing? I believe it would.
There are two potentially big issues with the use of statistics: 1. bad data, and 2. flawed interpretation.
Bad data can mean that the statistic is totally false – that the resulting data is just made up. This is frequently the case, and I’ll talk about a pervasive lie about self defense in a minute.
With statistics generated from studies or surveys, errors in data can creep in from sampling or how questions are posed. There are mathematical formulas that tell researchers how large a random sample of a population they must survey or test before their results cannot be attributed merely to chance. If you interview one guy in the United States and that guy happens to be Charles Manson – your data set isn’t going to represent the attitudes of the general population very well. The issue here is getting a truly random sample. Political polls aren’t random, they only poll people who are willing to answer polls – not necessarily those who may vote.
The other issue comes from the way questions are asked in surveys and polls. Look at these two questions: 1. Do you carry a knife? and 2. Do you ever carry edged tools for work or for other reasons including box cutters, knives, multi-tools, or the like? – People are likely to answer those two questions differently. People may be more likely to lie when there is a social stigma attached to the question. I’m aware of a survey done about butter consumption where rephrasing a question swung the results by around 15%.
In my opinion, one of the most commonly held misconceptions in self defense or martial arts is that all fights wind up on the ground, or 95%, or 80% or whatever number you’ve heard. Any number you hear in this regard is most likely completely made up (in other words – crap). This number has been actively promoted by people selling ground fighting systems. It sounds very authoritative, logical, and gives you something actionable to do.
There are only two studies that I am aware of. One study conducted by the LAPD before they designed their newest defensive tactics curriculum showed that in around 60% of hands-on use of force incidents, both the officer and suspect wound up on the ground. This is not only far less than 90%, but if you interpret use of force as an officer trying to take a suspect to the ground to facilitate cuffing – this sheds even more light.
Another study analyzed “street fight” videos uploaded to Youtube – which is not an analysis of all street fights – merely of the ones uploaded to Youtube. The results of this study showed that far less than 50% of fights analyzed wound up with both opponents on the ground. It also showed that the first person to go to the ground nearly always fared worse. Regardless, the term “street fight” does not imply self defense to me – just unsanctioned mutual combat.
So, let’s weigh the statistic on the scale of more good than harm. If you were to believe that all fights (100%) wound up on the ground, and so that you only learned ground fighting for self defense would that be a good thing or a bad thing? If the statistic is wrong, and you faced an opponent who knew how to keep off the ground, or multiple opponents, or someone with a weapon – that could be a very very bad thing.
So let’s say that we have a rigorously controlled study with good data, and we generate some statistics from that, what could go wrong? Well, even more insidious is the misinterpretation, or deliberately false interpretation of what the data represents. Misuse of statistics in this way is rampant in media and politics..
I once read an article on a newspaper’s web site that had the headline, “study shows that people who own guns are more likely to be murdered.” Really? No. The paper quoted a study that showed that the rate of gun ownership was higher in areas with a high murder rate. To even approach this interpretation, the study would have had to compare gun ownership amongst murder victims – it did not. Just as easily it could have been interpreted as people living in high crime areas feel the need to arm themselves for self-protection, but this would also be false as the study didn’t look at motivation for gun ownership.
So what is a good way to look at statistics?
Let’s take a statistic about sexual assault among college-aged women in the U.S. In some studies it is shown that there is alcohol consumption by either the assaulted or the criminal or both in 70% of reported cases of sexual assault. Without getting into the validity of the statistic, could we safely assume that it’s a good idea for young women to be careful both about their own alcohol consumption as well as being cautious about others around them who are consuming alcohol? Seems pretty reasonable. Could we assume that women who don’t drink have nothing to be concerned about? No, absolutlely not.
Universally, police and self defense experts warn people never to let a criminal take you to a second crime scene. After an abduction, the chances of an abductee surviving drop to almost zero. I do not believe this is in any way an urban myth and I believe the advice to be sound. We don’t know how many of the people who go missing every year are abducted, or how many survive that abduction. From the crimes we know about over time – this advice stands up.
Look to statistics in self defense with an air of skepticism, and think about the consequences of their interpretation.