DETECTING FOOD AND ACTIVITIES IN LIFELOGGING IMAGES

MyFood Automatic Monitoring of Eating Habits NTCIR: The first Lifelogging evaluation campaign

• Bahjat Safadi: Laboratoire d’Informatique de Grenoble • Rami Albatal: HeyStaks, ex. Insight Centre for Data Analytic

Bahjat.Safadi@imag.fr rami.albatal@heystaks.com @ramialbatal

MYFOODAUTOMATIC MONITORING OF EATING HABITS

IE Commercialisation Feasibility By Rami Albatal

Collaborators Cathal Gurrin, Bahjat Safadi

Rami Albatal

WHY

• Because there is a need for an easy way of writing and maintaining reliable and objective food diary. • This helps people in getting aware of their eating habits,

which leads into changes in eating behaviour and enhancing health conditions and an improved quality of life.

• But current solutions for food diary provide only manual input and monitoring services.

An automatic solution for eating tracking will save time and provide more objective food diary.

9/10/2014 3

Rami Albatal

THE POTENTIAL

• Healthcare Specialists / Nutritionists • MyFood automatically shows what food is taken and provides a

visual diary of food and eating habits (quantity, quality, ingredients, frequency… etc.)

• Slimming and Wellbeing • MyFood can offer a hassle free and objective food diary

solution. It allows also greater awareness by capturing photos for the personal food consumption.

• Food Market-Research • MyFood could detect consumed food and how the context of

consumptions in real-life settings (at work, in a social environment, on table, while walking, indoor, outdoor…etc.).

9/10/2014 4

NTCIR12-LIFELOGFIRST LIFELOGGING EVALUATION CAMPAIGN

Organisers

Cathal Gurrin, Rami Albatal, Liting Zhou (DCU, Ireland) Hideo Joho (Tsukuba University, Japan)

Frank Hopfgartner (University of Glasgow, UK)

Best Performance MRIM Team (Laboratoire D’Informatique de Grenoble, France)

Lead Participant: Bahjat Safadi

Rami Albatal

NTCIR12-LIFELOG

• First Lifelog test collection

• Components • A collection of domain-representative documents

(timestamp + images + semantic locations + movement + deep learning classification output).

• A set of queries (called topics) that are representative of the domain of application.

• A set of relevance judgments that may be complete or incomplete (pooled) and will identify the documents that are relevant to each query.

• Privacy is an essential feature

9/10/2014 6

Rami Albatal

NTCIR12-LIFELOG

9/10/2014 7

Autographer Moves App

Rami Albatal

NTCIR12-LIFELOG

Example of a Topic (query)

TITLE: Tower Bridge DESCRIPTION: Find the moment(s) when I was looking at Tower Bridge in London. NARRATIVE: To be considered relevant, the full span of Tower Bridge must be visible. Moments of crossing the Tower Bridge or showing some subset of Tower Bridge are not considered relevant.

9/10/2014 8

Rami Albatal

THANKS!

9/10/2014 9