Is My Activity Tracker Accurate?

Activity trackers like Garmin, FitBit, and the Apple Watch. Ever wonder if they are accurate in tracking calories burned?

According to a 2016 study from the Human Performance Laboratory at Ball State University, the answer is NO.

Researchers concluded that most activity trackers have a 10% (or higher) error rate. When counting calories, that’s a significant margin of error. For example, if your fitness tracker reported you expended 800 calories, your actual calorie expenditure could be + or - 80 calories.

A CLOSER LOOK AT THE STUDY

The study was designed to judge the accuracy and precision of activity trackers under normal circumstances.

Subjects took part in a structured activity protocol consisting of 11 activities, broken down as follows:

• Three sedentary
• Four household
• Four ambulatory/exercise

The activities were chosen by researchers from a list of 21 activities ranging from lying around on the couch to jogging on a treadmill. Activities were counterbalanced so that sex and age categories had approximately equal participation in the activities.

All subjects began their routine by lying quietly on a bed for 10 minutes. All other activities were performed for 5 minutes each, in order of generally increasing intensity.

The Results of the Study

During the study, 30 healthy men and 30 healthy women wore eight different types of activity monitors simultaneously while completing a 69-minute protocol.

The estimated calories burned reported by the activity trackers were compared to values obtained from an Oxycon portable metabolic system (basically a fancy device that monitors metabolic response). It’s worth noting that there is potential for error in the Oxycon device.

Below are the results from the study.

Figure 1: Mean absolute percent error when estimating energy expenditure for selected devices (Nelson. 2016).

Interestingly, the data shows that the devices were accurate, but they weren’t very precise. To put this in a metaphor, they all were using bows and arrows, they just didn’t hit the target. As you can see in the image above, most devices had around a 10% error rate.

WHAT IS THE BEST ACTIVITY TRACKER FOR COUNTING CALORIES?

Of all tested devices, only the results from the BodyMedia FIT, Fitbit Zip, and NikeFuel Band estimated within 10% of calories burned, which means a 90% confidence interval for that data.

I do not recommend trying to match your energy intake exactly to your energy expenditure. In my opinion, this can lead to obsessive and even dangerous behavior.

But, if you are interested in using your tracker this way, plan for a 10% + X% difference from your actual energy demands. And remember, even the Oxycon calorimeter used as a yardstick to judge the accuracy of the devices is not 100% accurate.

SHOULD I WEAR A DIFFERENT DEVICE FOR EACH ACTIVITY?

In that, it is also worth mentioning that the accuracy of the devices was activity and device dependent. For example, the Fitbit One produces the least error for stair climbing while the Jawbone UP24 is best at predicting your energy expenditure on the couch, as seen in the graphic below.

(Nelson, 2016)

Accordingly, you could argue that you'd have to wear a certain device for a certain activity, e.g. (a) the Fitbit One, when sitting around (13%), working in the household (27%), taking the stairs (11%), jogging (22%) or cycling (43%) [note: on absolute terms, the error of the Fitbit for being sedentary is still lower than with the device from Jawbone], and (b) the Jawbone UP24, when you're simply walking around.

Of course, since even that wouldn't be 100% accurate, it would be dumb to buy multiple activity trackers, wouldn't it?

DOES A 10% ERROR RATE IN MY ACTIVITY TRACKER EVEN MATTER?

The answer depends entirely on how you use your fitness tracker.

If you use your activity tracker to count calories, the answer is YES! 10% is significant.

Let me explain.

If you match your caloric intake exactly to the estimated caloric expenditure on the device, an error rate of 10% is significant enough that you will always lose or gain weight.

For example, let’s say your fitness tracker estimated you burned 500 calories during your usual workout routine.

For simplicity, let’s say you are trying to maintain your weight (calories consumed=calories burned).

• 10% of 500 is 50 calories.
• Assuming your daily workout is similar most days, if your calorie counter consistently estimates you burned 10% more calories than you actually burned, you’re consuming an extra 350 calories a week.
• Over a month, you’d consume an extra 1,400 calories.

Of course, there are too many variables at play to take my calculation above seriously.

• What if your tracker is +100 calories today, -20 tomorrow, and -10 the next?
• How does the amount of sleep you get and the kind of calories you consume affect the results?
• How do different kinds of exercise over different durations change things?

OBESSSING OVER CALORIES CAN LEAD TO OTHER PROBLEMS

My point is that if you are that determined to count calories, you’ll need to find a more precise and accurate way of calculating your energy expenditure. If you blindly match your diet to the data your activity tracker provides, you’ll never achieve reliable results.

With that being said, our body is a biological machine that works according to a set of several (complex) equations. Therefore, the whole idea of a “quantified self”—as awesome as it may seem to the average control freak—is simply not possible for most people that only have access to consumer grade equipment.

And frankly, whatever reward you get from counting calories is outweighed by the risk of exercise addiction, body dysmorphia, obsessive-compulsive disorder (OCD), and a whole cast of other problems that come with an unhealthy relationship to your body.

IS MY FITNESS TRACKER GOOD FOR ANYTHING?

I’m glad you asked!

Activity trackers are useful when the data is used as a benchmark to help you stick to your weekly exercise goals.

For example, let’s say that on Friday you look at the data which suggests you’ve been about 20% less active this week than your goal. You then decide to head to the gym to get caught up.

In that way, activity trackers are a tool to hold yourself accountable; a qualitative or semi-quantitative device that helps you make a judgment call about whether you’re living the lifestyle that suits you best.

References

• Nelson, Benjamin N; et al. "Validity of Consumer-Based Physical Activity Monitors for Specific Activity Types ED." Med Sci Sports Exerc (2016): Ahead of print.