Cybercrime Stats are Vastly Exaggerated

Last week, I shared my take on why you really shouldn’t worry too much about cyber war. Here are some thoughts on the related subject of cybercrime.

Two facts that will challenge your view of the world: 1) Cybercrime isn’t as big a problem as we think it is; and 2) Many of the terror threats the government says it has protected us from were products of its own creation, which demonstrates how good the public and private sectors are at creating imaginary hobgoblins to scare us with and how happily we go along with it.

Let’s take a closer look at cybercrime. Estimates place related annual losses between billions and a trillion dollars. That easily makes cybercrime one of the fastest growing industries on the planet. There’s only one problem: As researchers Dinei Florencio and Cormac Herley point out, “It turns out, however, that such widely circulated cybercrime estimates are generated using absurdly bad statistical methods, making them wholly unreliable.”

By and large these estimates are based on surveys of consumers and businesses. The pollsters take the survey answers and (hopefully) do some fancy math to pull conclusions that they then share with the general public. Unlike political polls, these cybercrime surveys ask for specific numbers not just preferences such as, “How much more do you like this dopey guy than that dopey guy?” If 37 percent of the public say they like dopey guy A, then you actually know something. However, according to Florencio and Herley:

“[I]n numeric surveys, errors are almost always upward: since the amounts of estimated losses must be positive, there’s no limit on the upside, but zero is a hard limit on the downside. As a consequence, respondent errors — or outright lies — cannot be canceled out. Even worse, errors get amplified when researchers scale between the survey group and the overall population.”

And there’s the assumption that the group being surveyed is large enough to be representative of the population as a whole, which is clearly an assumption that should not be made when it comes to these statistics.

“The cybercrime surveys we have examined exhibit exactly this pattern of enormous, unverified outliers dominating the data. In some, 90 percent of the estimate appears to come from the answers of one or two individuals. In a 2006 survey of identity theft by the Federal Trade Commission, two respondents gave answers that would have added $37 billion to the estimate, dwarfing that of all other respondents combined.”

I know of a highly technical term that experts apply to actions like this: Lying.

Crime statistics–physical or cyber–are almost always dubious. At their best they only measure reported crimes. In the physical world we know crimes are under-reported because victims often don’t report them to the authorities. The victims don’t think it will do any good or, in crimes like rape, they think it may be too traumatic.

Cybercrime statistics are even more problematic because sometimes the victims don’t even know they’ve been hacked. And, again, that is assuming the statistics are actually legit to begin with. (Yes, Group-IB, I am referring to you again.)

In our next installment we will take a look at how the FBI has taken bogus surveys one step further by creating actual bogus threats. Stay tuned.