What a Mean Hides: How to Read an Accuracy Figure Without Being Misled

We cite accuracy figures constantly on this desk, so it is worth explaining what they are and what they are not.

What the number is

Usually a mean absolute percentage error across a set of reference meals. Each meal is weighed and its true content computed; the app estimates it; the difference is recorded as a percentage; the average of those percentages is the figure.

So ±1.1% is a statement about a meal set. Your dinner was not in the meal set.

Three things a mean hides

The spread. A mean of 1.1% is compatible with most meals being very close and a few being badly wrong. In both studies we cite, individual plates missed by considerably more than the headline figure.

Composition. An estimator strong on flat plated food and weak on composite dishes scores differently depending on what the tester cooked. The meal set’s contents matter as much as its size.

Failure handling. What did the study do when an app refused to estimate? Excluding those cases flatters an app that declines often. This is buried in methods sections and it changes results.

The three questions to ask

Who measured it? If the answer is the company selling it, you have a marketing figure. Not necessarily false; not a measurement that should move your decision.

Has anyone reproduced it? A single independent measurement establishes the figure is not self-reported. Reproduction by a second unrelated party on a different meal set establishes the test design was not doing the work. Very little clears the second bar.

What was measured? Photo estimation and manual entry are different operations with different error profiles. Figures for the two are not comparable, and an app can be excellent at one and ordinary at the other.

The translation

On a 2,000 kcal day

~1% → ±20 kcal. Below the noise of everything else in your week.

~5% → ±100 kcal. Detectable over a month, not over a day.

~12% → ±240 kcal. The size of a typical deficit.

That last line is the whole practical case. At the loose end, the instrument moves as much as the thing being measured.

And the error the figures do not cover

Your portion estimate. In our own weighing week it exceeded every app’s estimation error.

Read these numbers as descriptions of instruments, not as promises about dinners.

Questions we get asked

What does MAPE mean in a calorie app study?

Mean absolute percentage error — the average of the absolute percentage differences between the app's estimate and the true value across a set of reference meals. It is a statement about that meal set, not a guarantee about any single meal, and it says nothing about the spread. A mean of 1.1% is compatible with most meals being very close and a few being badly wrong.

How should I interpret a 10% error band day to day?

On a 2,000-calorie day, roughly 200 calories of uncertainty — about the size of a typical daily deficit. That is the practical consequence: at the loose end of this category the measurement error and the effect you are trying to measure are the same magnitude, so no single day or week supports a conclusion.

Why does test-set composition matter?

Because photo estimation performs very differently on flat plated food than on deep containers, where depth is not recoverable from a single overhead image. A study built mostly of plated meals will report better numbers for every photo system than one built mostly of bowls, and neither is wrong — they measure different populations of meals. An aggregate figure with no stated composition is not comparable to anyone else's.

Caleb Ostrowski

Editor · Stronger by Math

Coaches lifters and writes the arithmetic down. Nine years of client logs, most of them unglamorous. Buys every app on this site at retail and cancels most of them. No affiliate links anywhere on this desk.