The Confidence Label Was Never a Substitute for a Scale

Researchers at the American Society for Nutrition’s annual meeting last month ran the test every calorie-counting app, Mr BITE included, has been quietly hoping nobody would run. Olivia Charles and Aaron Hengist took advantage of an ongoing study at the NIH Clinical Center, where every meal is prepared in a metabolic kitchen and every ingredient is weighed to a tenth of a gram before it’s plated. That gave them something almost nobody testing a food app ever has: a real, exact answer for what a meal actually contained. They photographed 102 of those meals and ran the pictures through four popular apps, MyFitnessPal, Lose It!, Cal AI and Appediet, and compared what came back against the ground truth sitting right there in the kitchen’s own records. All four apps underestimated the meals, by 250 to 345 calories on average and by about 30 grams of fat. That’s not a rounding error. That’s close to a third of a typical meal’s calories, missing, with a confident number sitting where the missing calories should be.

I built the version of this feature that lives in Mr BITE, so I read that abstract with a specific kind of discomfort. Point your phone at a plate, and our backend runs the same kind of estimate those four apps run: a vision model looks at the photo and guesses calories, protein, carbs and fat. We already know this number is a guess, which is why the estimate carries a confidence tag, high, medium or low, and why the follow-up message we send our own coach explicitly says to treat it as a visual estimate and call out uncertainty. We even strip the model’s own hedge words, the “about” and “approximately” it likes to put in front of a portion size, because a hedge that never changes anything downstream is just noise sitting in front of a number. On paper, that looks like an app that already understands photo estimates aren’t the same kind of measurement as a barcode scan or a gram entered off a kitchen scale.

Here’s what the NIH study actually forced me to notice, though: a confidence label generated by the same system that produced the guess isn’t a check on the guess. It’s the guess’s opinion of itself. “High confidence” in our pipeline means the vision model felt sure, not that the number lands within some known distance of the truth. Those two things might correlate. We don’t know that they do, because we have never done what Charles and Hengist did: photographed a weighed, known meal and compared what came back. Every other number in Mr BITE that we’re proud of, the grams a barcode scan pulls from a product database, the macros a database entry carries, sits on top of a source we can point to and say here’s where that number came from, and here’s why it’s right. The photo estimate sits on top of a model’s self-report, dressed up in the same interface as everything else.

None of this means photo logging is useless, or that our confidence tag is theater. A hedge word that never changes behavior is worse than no hedge at all, and at least ours is trying to say something true about the estimate underneath it. But trying to say something true and actually being calibrated are different claims, and only one of them can be checked by running the same test the NIH kitchen ran: weigh real meals, photograph them, compare. We haven’t run that test on our own pipeline yet. Until we do, “high confidence” is a badge we’re offering in good faith, not a number we’ve verified. That’s the gap this study actually closed for me, not a criticism of four other apps, a to-do list item for our own.

BITE of the Week: When the Coach Said “Done” and Hadn’t Done It

Version status: 1.10.0 is cut, marketing version bumped from 1.9.0 on August 22nd, with EAS build numbers assigned the same day (iOS build 104, Android build 135). It carries recipe version families and an in-plan personalisation flow, plus the write-receipt guard below. 1.9.0 remains what’s actually live on both stores until 1.10.0 clears store review and finishes rolling out.

The headline fix closes a gap that had been live since July 10th. Nine times across two customers, Mr BITE’s coach replied with some version of “Done, added it to Monday’s dinner” without making a single successful write, an honest-sounding sentence describing something that never happened. The new guard checks every claimed-write reply against the turn’s actual tool calls, and any receipt claiming success with no successful non-read tool call behind it now gets swapped for an honest correction instead. It’s built narrow on purpose: anything genuinely ambiguous, an unknown tool, an unparseable result, a device-only action, stands the guard down rather than risk overwriting a true receipt with a false correction, which the team judged the worse of the two mistakes.

Two more fixes from the same sweep are worth knowing about even though they’re invisible day to day. The AI health-check watchdog had been silently dead since July 25th: it was missing from the one config list that lets a function skip gateway JWT verification, so its own cron pings were rejected as unauthorized before the monitoring code ever ran, and the incident table it was supposed to populate sat at zero rows the entire time, which looked like a clean bill of health rather than a monitor that had never once actually run. It now upserts a heartbeat on every pass, so an empty incident table means checked and fine, not who knows. And Google Health sync was quietly storing “connected” for accounts it could never actually sync (the provider requires a linked Fitbit profile, and every one of four non-founder accounts lacked one) instead of recording the real failure and giving up gracefully; that’s fixed too, with the raw provider error strings that used to leak straight into the user-facing progress screen now routed through friendlier copy.

Three Quick Bites

1. Four popular AI food-photo apps underestimated meals by roughly a third when tested against weighed, known meals at the NIH Clinical Center. Researchers Olivia Charles and Aaron Hengist photographed 102 meals prepared in a controlled metabolic kitchen, each weighed to the nearest 0.1 gram, and ran the images through MyFitnessPal, Lose It!, Cal AI and Appediet: all four underestimated calories by 250 to 345 kcal and fat by about 30g per meal on average. The work was presented at NUTRITION 2026’s President’s Oral Session and hasn’t yet completed full peer review. It’s the study behind the essay above. ScienceDaily

2. The FDA and USDA sent the federal government’s first proposed definition of “ultra-processed food” to the White House budget office for final review this month, and food company CEOs have spent the weeks since trying to get it delayed. The white paper, which HHS Secretary Robert F. Kennedy Jr. has tied to the roughly 60% of the American diet already made up of ultra-processed foods, carries no legal force on its own, but executives and trade groups have been calling and writing the White House directly, arguing that a formal definition opens the industry to litigation and higher costs. Bloomberg

3. A four-week randomised trial found adults who added a daily glass of 100% fruit juice or a smoothie to a low-fruit-and-veg diet had depression scores 2.52 points lower, on a 27-point scale, than a group that increased produce intake without it. The Newcastle University-led trial, published in the British Journal of Nutrition and funded by the Fruit Juice Science Centre, is real and peer-reviewed, but small, short and industry-funded, and the authors call it promising rather than proof, with an added caution for anyone managing blood sugar. ScienceDaily

Tool of the Week: ASA24

Since this issue is about the gap between an app’s estimate and a meal’s real, weighed contents, this week’s pick is the free tool researchers themselves reach for when they need a dietary record they can trust. ASA24, the Automated Self-Administered 24-Hour Dietary Assessment Tool, is built by the National Cancer Institute for exactly the kind of studies whose numbers end up cited in the rest of this newsletter: a free, web-based system that walks a respondent through a structured 24-hour recall or multi-day food record and automatically codes it against USDA’s own food and nutrient data, the same category of ground truth the NIH kitchen study above was built around, just self-reported instead of weighed in a lab. For readers: it’s a useful model for what a genuinely careful food record looks like, structured prompts for portion size and preparation method instead of a single photo and a guess. For builders: the Researcher Site gives you the same instrument NIH-funded studies use to generate the dietary data that ends up validating, or this month, not validating, apps like ours. ASA24 at NCI. Free, NIH-built, registration required for the researcher tools.

Thanks for reading. If you’d rather your coach told you when it doesn’t actually know, Mr BITE is free on iOS and Android.

Collins, building Mr BITE