Understanding the Accuracy of Flight Data

(Note: This article was originally published on July 9, 2021).

In my last post, I looked at “bad data” and some of the ways that we could deal with it.

In this post, I would like to take a look at some of the “limitations” of flight data and set expectations for how accurate the values that you see in the flight data can be. This is different than “bad data”. In this case, the data is as accurate as the designers intended – it just may not be as accurate as you are expecting and it is important to have a good understanding of these limitations.

Two main factors that affect parameter accuracy are Sample Rate and Resolution. Sample rate is probably easiest to understand, so let us start with that.

When I refer to Sample Rate, I simply mean how frequently the parameter is recorded, or “sampled”. Let us use Air Ground (or Weight On Wheels) as an example. In most aircraft, Air Ground is recorded once per second, or at 1 Hz. But some aircraft only record it once every 4 seconds, or 0.25 Hz (Note: NOT 4 Hz, which would be 4 times a second).

So, if you are looking at Air Ground in your flight data and you identify the point in the data where the Air Ground switch changed from Air to Ground, you could be off by up to 4 seconds. At 120 kts groundspeed, that’s over 800 ft! That would make it difficult to pinpoint the point of touch down if you were to rely only on the Air Ground switch.

Again, this is not “bad” data. Everything is working as intended. This is just a limitation of the system that we need to be aware of before coming to any conclusions about the flight. Avionics engineers only have so much space and “bits” to work with, especially in older aircraft.

For them, it is a balancing act determining what needs to be recorded more frequently and what can be recorded less frequently. They could record Air Ground at 4 Hz, but that might mean something like Vertical Acceleration (considered a more important parameter) could only be recorded at 1 Hz (rather than 8 Hz or 16 Hz).

Keep the sample rate in mind when you are studying your data – especially those parameters that are not updated frequently. If you need help determining sample rate, check out our YouTube video or drop us a note. It is outside the scope of this blog, but we do plan to offer a future course on FRED files and data frames which should make it clear.

Resolution

The other property of parameters in your flight data that can affect its accuracy is the resolution at which it is recorded.

When I refer to resolution, I am referring to what is basically the minimum value that each raw “count” of data can represent. For example, Altitude could be 1 ft/count. Airspeed could be 0.5 kts/count.

To better understand this, we are going to need to do a little bit of math, but do not worry – it really is not that complicated and I will keep it simple.

On the typical flight data recorder or quick access recorder, the data is stored in “bits” that can either be a 1 or a 0 (if you already are a bit expert, you can skip the next few paragraphs). This is also called “binary data”. This is great for computers and electronics, but we are used to working in “decimal” where the digits go from 0 to 9. We can refer to these decimal values as the “count”.

Before I throw out a formula, let’s take a look at a few binary numbers and their decimal equivalent:

It looks pretty straight-forward, and it is, but it can still make your head hurt when you first start using it.

So, why am I showing you this?

To understand the resolution of a parameter, we need to know the “raw count” range based on the number of bits allocated for that parameter.

For a binary number, the range is easily expressed as 0 to 2^n – 1. That hat (^) symbol is meant to be “to the power of” and “n” is the number of bits.

Looking back at the table above, we have 3 bits available so the raw count range is 0 to 2^3 – 1, or 0 to (8 – 1), or  0 to 7.

Ok. With that out of the way, we can get back to looking at how this affects the accuracy of our data. Let us take Pressure Altitude as an example. We will ignore the possibility of negative altitudes for now and consider that we may fly our aircraft at a Pressure Altitude of anywhere from 0 ft to 40000 ft.

Let us assume we have 12 bits to work with. This is the length of an ARINC 717 “word”, but that’s for another blog (check out or YouTube video for more information on Words, Subframes and Bits).

With 12 bits, we can have a workable range (or counts) of 0 to 2^12 – 1 or 0 to 4095. We can now determine our resolution of 40000ft/4095 counts = 9.77 ft/counts or about 10 ft/counts.

That is not bad, but it is not great either. There are “tricks” that the avionics engineers can do to improve this resolution, such as combining “words”, but that is outside the scope of this blog. Just be aware of the resolution of your data and how it can be determined.

Resolution becomes particularly important when looking at positional data such as Latitude and Longitude. These parameters can have an accuracy of +-200 ft or so, which is fine when you are trying to determine where the aircraft is over the country, but it is very limiting if you want to determine where the aircraft touched down.

Latitude and Longitude data that is collected from an inertial source (Inertial Navigation System – INS or Inertial Reference System – IRS) is even worse. This data is typically very accurate at departure, but becomes progressively less accurate as the flight continues. On longer flights, it may be nearly impossible to determine which runway the aircraft landed on, let alone where it touched down.

Again, this does not mean that the data is “bad”. This is just a limitation of the system.

As a data analyst, it is your responsibility to understand these limitations and to make sure that they are effectively conveyed to your audience so that they understand these limitations.

Showing a flight animation of an aircraft landing on the grass beside a runway is going to do nothing for your credibility, even if that is exactly what the INS positional data showed!

Let’s keep in touch

Sign up to get notified of new blog posts, videos or other news and information related to flight data.