When is a step a step? Science examines when activity trackers are worth it

7,643 steps. Which means I still need another 2,357 to reach my daily goal. A lot of people feel motivated to drag themselves back off the sofa when they have a smartwatch or tracker around their wrist. That alone gives these devices a purpose. But when it comes to how valid the data on the display is, that’s another question entirely.

The number of steps is usually recorded by an accelerometer. Using three axes, it registers how you move around the room. This post outlines the principle in more detail. The tracker then harnesses movement patterns to conclude when you must have taken a step. When in actual fact, you might just have been sitting at the kitchen table gesticulating wildly. If you want to calculate in absolute terms, you have to ask yourself: how precise is the device at the end of the day? For example, my watch always adds on a few extra metres when I’m jogging if it’s measuring the distance travelled using a sat nav.

Measurement accuracy isn’t just something that concerns sports enthusiasts. In fact, it’s been the topic of many scientific publications. Obviously using different devices and approaches and generating varying results. These all stand alone in their own right. That is, until someone takes the time to do a systematic review and filter out which are comparable and which findings can be inferred from it. A similar paper has just been published on the topic of fitness data retrieved from wrist-wearable trackers.

65 studies want to know the specifics

Of the wave of publications in large databases, a total of 65 studies were sifted out that dealt with measurement accuracy under comparable conditions. Participants were said to represent the average population in terms of gender, age and level of training. The studies measured accuracy using various values that common fitness trackers record, including steps, activity levels, intensity, energy consumption, heart rate, distance and speed. But, when it came down to it, the data situation was only good enough in three areas where it could be examined in more detail. Steps, heart rate and energy consumption were the points checked most often.

Individual studies demonstrate good results for individual trackers. It’s more meaningful when a number of researchers independently come to similar conclusions. Meanwhile, if the values differ widely, it shows how crucial the test set-up is. Researchers’ favourite value is the Mean Absolute Percentage Error (MAPE). In other words, the percentage of absolute errors. This is significant in the cross-study environment, as it puts the mistake in the context of the total amount. To give you an example, being 50 steps off the mark is a different kettle of fish if we’re dealing with a total of 100 rather than 10,000.

There’s only praise for Fitbit

Over the course of 31 studies, 72 different devices from 29 brands decided whether or not to count a movement as a step. In some cases, you can take this literally because some industrious people looked through video recordings and pressed a hand counter after each step. In other cases, the recount was done by a machine. The review only highlights two Fitbit models. That’s partly because they were used in 20 studies – in other words, very often. And partly it’s because they displayed a MAPE less than 25%, leading the authors to declare: «The Fitbit Charge and Fitbit Charge HR were consistently shown to have a good accuracy for step counts.»

Sports watches + Smartwatches

EUR159,–

Fitbit Charge 5

14.73 mm, Steel, One size

119

Sports watches + Smartwatches

EUR159,–

Fitbit Charge 5

14.73 mm, Steel, One size

119

Fitbit Charge and Charge HR are certified to have good accuracy, although the range of results is still relatively wide. That doesn’t translate as good news for other models that don’t perform as well. But it is good news for you. It doesn't really matter what your watch shows in the evening. Displaying 10,000 steps? It could just as well be 11,000 or 9000, which makes all the talk around the «right» daily goal seem even more absurd.

It’s impossible to say if trackers systematically capture too many or too few steps. Sometimes it goes in one direction; other times, it goes too far the other way. It’s a similar story with heart rate data. You’ll find that Apple Watch authors consider a MAPE less than 10% to be a good value. When it comes to energy use, on the other hand, that’s when all manufacturers are in the dark. Researchers point out that there’s not a single device in this category that stands out with realistic data. That means casting a glance at your calorie counter is what we always suspected: just a waste of energy.