Just Tell Me
Food labels spread essential information across small print, multiple panels, and barcodes. A blind or low-vision shopper may know exactly what they need to check but still cannot tell where on the package that information appears.
Why this matters
Ingredients, allergens, nutrition facts, and expiration dates affect everyday health and safety. A person should not have to visually find the correct side of a package before an app can help read it.
- Current idea
- Just Tell Me is a focused food-label assistant within the AIVIS Vision mobile experience. A user asks a specific question—such as whether a product contains an allergen, what the ingredients are, what the nutrition facts say, whether an expiration or best-by date is visible, or what product a barcode identifies—then points the rear camera at the package or selects up to six photos. Because food information is often spread across different sides of a package, the app combines evidence from multiple camera angles or selected images, then gives a short spoken and on-screen answer supported only by what is visible. It can read ingredient lists, identify potential allergens from visible ingredients, summarize nutrition labels, locate expiration or best-by dates, and recognize barcodes. For supported food items, verified product information from Open Food Facts may be used when a reliable barcode or product match is available. It must never invent an ingredient, allergen, nutrition value, or date that is not visible or supported. Users should verify critical allergy and food-safety decisions because AI analysis can be incomplete or incorrect. Simple positioning guidance such as move closer, hold steady, or slowly turn the package helps the user capture what the app needs. Other visual questions, such as finding a Start button, reading a price, or comparing two items, are secondary uses.
- Accessibility notes
- VoiceOver-compatible controls and announcements; spoken camera guidance and answers; optional haptic feedback; large touch targets; high contrast; short action-oriented instructions; camera guidance that does not rely on visual landmarks; multiple-view analysis so the user does not need to find a particular panel first. Users can type a question or use iPhone keyboard Dictation. The app uses text-to-speech but does not record audio. History, Saved Questions, and accessibility preferences are stored locally on the device. No account or login is required.
- What was learned
- Never make a blind user find the label before the app can help read it. Guide first, combine evidence across useful views, and answer only what the evidence supports—especially for food safety and allergies, where an incorrect or invented answer can cause real harm.
Development updates
Progress in public
Core food-label analysis is implemented: ingredient-list reading, allergen identification from visible ingredients, nutrition-label analysis, expiration and best-by date detection, barcode and product matching, multi-view evidence combination, spoken positioning guidance, History, Saved Questions, and accessibility settings. The project is being prepared for a successful iOS release.
Feedback
Have thoughts on this project?
Email openindependence@gmail.com to share a related barrier or suggestion. Attribution is optional.
Get in touch