Building a Revolutionary AI-Driven Nutrition Tracking Platform that Transforms Diet Management

Client: PASSIO.Inc
Industry: Healthcare & Wellness
Core Tech:
- AI/ML
- Computer Vision
Services Provided:
- AI/ML Development
- Data Science Solution
- Computer Vision Solutions
- Mobile App Development
- Strategic Consultation

A Vision for Healthier Lives

Studies say that around 95% of people following strict dieting methods to lose weight often end up making dietary mistakes. This shows the struggle of individuals to track their daily nutritional intake when they are trying to make healthy choices. Our customer, Passio.AI, saw an opportunity to change the narrative. They leveraged AI to not only simplify nutrition tracking but also transform it into a smart, data-driven process.

Identifying the Challenge

With a vision to make an AI solution for each vertical, our client founded Passio.AI, backed by a $3M investment from SRI Ventures to tackle a universal problem: people want to eat better, but they often lack the right tools or guidance.

Fast-forward a few years, and Passio.AI’s Nutrition AI has evolved into one of the most robust food-recognition platforms in the world. Boasting the world's largest visual food database and partnerships with industry leaders like MyFitnessPal and Elevance Health, Passio.AI has turned the once-tedious chore of nutrition logging into a seamless, intelligent experience.

Despite the early success, Passio's Nutrition AI faced serious challenges in:

Recognition Accuracy

Foods come in endless shapes, sizes, and cultural variations. The AI was lacking that intelligence to correctly identify different food items, from the bowl of quinoa to a plate of sushi.

Portion Size Estimation

There was also a lack of inaccuracy in estimating the food portion size, specifically when dealing with layered or complex dishes.

Processing Speed & Performance

The solution was taking too long to detect the food item and fetching details out of it, creating poor user experience (UX).

Cross-Cultural Food Diversity

It was also lacking in recognizing the global array of cuisines apart from Western dishes.

Integration Hurdles

The Nutrition AI needed to work seamlessly across Android and iOS devices while ensuring a consistent experience.

Nutrition Logging

The solution was lacking capabilities to log the nutritional value of food items in a proper format.

Doctor's Report Generation

There was no option to generate a report of patients’ nutritional intake chart in PDF form to share it with them, and that too, in real-time.

A Four-Step Training Pipeline Used to Optimize The Solution

To solve these challenges, Passio and MindInventory started the R&D together to train the AI model rigorously. At its core, the mission was to create an SDK that would make nutrition tracking both accessible and accurate. The process looked something like this:

Data Collection
- Created a custom dataset of real-world food images, captured by food lovers in various lighting and plating conditions.
- Leveraged publicly available datasets, like Google’s Open Images Dataset and Open Food Facts, to expand the model’s understanding of global cuisines.
Data Preprocessing & Augmentation
- Did image augmentation (like image rotation, scaling, and brightness adjustments) using OpenCV & TensorFlow to advance the photo logging.
- Empowered the model with OCR capability (used Tesseract Open Source OCR Engine and Google Vision AI) to extract nutrition information from food nutrition labels.
Model Selection & Training
- Leveraged computer vision models, including CNN, YOLO, ViTs.
- YOLO for high-speed object detection.
- CNN for accurate classification of single or multiple foods.
- ViTs for advanced object recognition for the vast variety of food items.
Model Optimization for Mobile Deployment
- Used TensorFlow Lite & CoreML to shrink the model’s size and boost speed.
- Leveraged quantization & compression method to make the model run smoothly on devices with limited processing power.

From Brainstorming to Co-hosting Innovation Workshops

Passio.AI hired us to be their external technology consultant, but we ended up as an extension of their in-house team. We collaborated at every stage to refine ideas and align strategies. Our mobile platform development side contribution included:

Made Nutrition Logging a Click Away By Implementing Photo Logging

Our engineers helped the client to build a Nutrition AI SDK that can extract information about food nutrients from micros to macros by implementing advanced features that extract and log information via photo logging (just by scanning the food item through the phone camera, as well as clicking the photo of it) and barcode scanning.

Nutrition AI Advisor Offering Personalized Advice

The system’s AI advisor goes beyond basic interactions. It offers personalized nutrition advice and meal plans with a tracking feature that not only boosts engagement but also becomes an invaluable tool for health apps.

Hands-Free Tracking with Voice Logging Feature

We integrated advanced voice logging functionality, enabling users to log their meals effortlessly. This hands-free approach enhances user convenience and streamlines the nutrition tracking process—ideal for busy lifestyles.

Modified the SDK to Help Passio Team Cater to Their Clients

We revamped the SDK to elevate Passio.AI’s internal capabilities while equipping them to better serve their clients. Leveraging our expertise in Kotlin, Swift, Flutter, and React Native, we enhanced the SDK, feature APIs, and components for iOS, Android, and Web to ensure seamless integration with third-party fitness solutions.

Advanced Text Search & Custom Food Creation

Our solution includes a robust text search feature with OCR implementation that allows users to easily extract nutrition information from food package labels.

Implemented A Feature Generating Doctor's Report in PDF

Recognizing the importance of healthcare professionals receiving accurate, shareable data, we introduced a feature that automatically generates PDF doctor’s reports. This functionality streamlined communication between patients and healthcare providers, offering a clear snapshot of daily nutrition logs and progress.

Impacting Lives with Innovation

Passio.AI’s Nutrition AI is the solution of bold vision, deep research, and a relentless drive to solve real-world problems. By combining on-device edge AI with scalable cloud services, we’ve helped them create a platform that is fast, accurate, and widely adaptable.

Today, Passio’s Nutrition AI recognizes 2.5 million food items, 1 million food package labels, and thousands of unique food items.
It's food recognition accuracy has improved by 97%.
Helping nutritionists and doctors significantly decrease the margin of error in patient diets, the current improvement is at 27%.
Helping healthcare providers save 75% of time in attending patients’ diet history.