Numbers Don't Lie?

This year’s NBAA-BACE includes a session with the provocative title “The Truth is, Humans Lie...But, Numbers Don’t.” Perhaps the title comes from the more famous version, “Figures don’t lie, but liars do figure.” The quotation comes from Carroll D. Wright, a U.S. government statistician, while addressing the 1889 Convention of the Bureaus of Statistics of Labor.

The session will focus on business analytics for fixed-base operators (FBO) and how to leverage analytical tools to improve business. While I am sure the intention is to provide a glimpse into valuable statistical tools that can improve FBO operations, I think it is important to look at all quantitative analysis with a healthy dose of skepticism. Three hypothetical examples of statistical malpractice can be helpful.

Acme FBO’s Hangar Rash Up By 100%!

Percentages lie. Let’s say your FBO of choice suffered a 100% increase in hangar rash last year. That would be an FBO to avoid, right? If your FBO handled one million aircraft movements in each of the last two years and had run aircraft into each other 50 times the first year and 100 times the second year, yes, you might want to avoid them because 100 is a big number. But what if the numbers were 1 and 2? The rate for the first year was one in a million for a rate of 0.001% and for the second year was 0.002%. The rate went up 0.001%, but it also doubled.  While the 100% claim is true, it certainly doesn’t tell the truth.

Acme FBO Fuel Price Hikes Create Greater Demand!

Correlation or causation? A good economist will tell you increasing prices should decrease demand, so it would be tempting to say the FBO fuel business is exempt from the laws of economics. But in this example, did the hike in fuel prices cause greater demand, or were both in response to increased flight operations? The classic correlation-or-causation example is that increased consumption of ice cream causes heat stroke, when it is hot weather that causes both.

Acme Rated Top FBO In Local Survey!

Let’s say you have two FBOs at your airport, Name Brand Inc. and Acme LLC.  The airport conducts a survey of 700 crews from each FBO flying midsize cabin and large cabin aircraft.  The survey reveals news both FBOs can crow about. Most midsize cabin crews prefer Name Brand, most large cabin crews also prefer Name Brand, but most crews prefer Acme. How is this possible?

This is known as Simpson's Paradox, named after Edward Hugh Simpson, a World War II British codebreaker. The phenomenon results when a factor assumed to be associated with the comparison in question is not. In our example, the airport assumed asking an equal number of crews using both FBOs will provide ample data points and that breaking them into midsize and large cabin crews would be useful.  Just because Acme wins the survey 83% to 78% doesn’t mean most crews prefer Acme. A more meaningful statistic would be to say more midsize cabin crews use Acme and more large cabin crews prefer Name Brand, but even that would be questionable. If it took a year to accumulate 700 votes for Acme, while Name Brand did that same business in a month, the entire survey should be questioned.

So, yes, people can lie. But so can numbers. The method of obtaining the numbers can be more important than the numbers themselves. UK Prime Minister Benjamin Disraeli may have said it best in 1891, “There are three kinds of lies: lies, damned lies and statistics.”  Even if the numbers seem to be telling the truth, you still need to apply a great deal of common sense. If you find one FBO leads as the top choice for most of your peers, it could be that the promise of a free steak or apple pie influenced the vote more than fuel prices and the quality of the line service.