FOOD SELFIES A DIETARY AID? Imagine being able to post an image of your food product on social media, and artificial intelligence being able to pick out the ingredients you have used in the recipe. In the future, smart food recognition systems will be able to do just that from an image, meaning consumers can figure out exactly what’s in their food when they don’t have explicit nutritional information. In fact, the technology is already here in the form of Pic2Recipe, with MIT researchers saying online food images provide valuable insights into what people are eating and why. In a new paper with the Qatar Computing Research Institute (QCRI), the team trained Pic2Recipe to look at a photo of food and be able to predict the ingredients and suggest similar recipes. “In computer vision, food is mostly neglected because we don’t have the large-scale datasets needed to make predictions,” MIT researcher Yusuf Aytar says. “But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences.” The web has spurred a huge growth of research in the area of classifying food data, but the majority of it has used much smaller datasets, which often leads to major gaps in labelling foods. Three years ago, Swiss researchers 42 AUGUST 2017 created the ‘Food-101’ dataset and developed an algorithm that recognised images of food with 50% accuracy; and City University in Hong Kong has more than 110,000 images and 65,000 recipes, but all are Chinese cuisine. MIT researchers combed websites like All Recipes and Food.com to develop ‘Recipe1M’, a database of more than one million recipes that were annotated with information about the ingredients in a wide range of dishes. They then used that data to train a neural network to find patterns and make connections between the food images and the corresponding ingredients and recipes. Given a photo of a food item, Pic2Recipe could identify ingredients like flour, eggs and butter, and then suggest several recipes that it determined to be similar to images from the database. “You can imagine people using this to track their daily nutrition, or to photograph their meal at a restaurant and know what’s needed to cook it at home later,” MODUL University Vienna assistant professor Christoph Trattner says. “The team’s approach works at a similar level to human judgement, which is remarkable.” The system does particularly well with desserts like biscuits or muffins, since that is a main theme in the database. However, it has difficulty determining ingredients for more ambiguous foods, like sushi rolls and smoothies. It is also often stumped when there are similar recipes for the same dishes, such as lasagne. “In the future, the team hopes to be able to improve the system so that it can understand food in even more detail,” CSAIL graduate student and lead author Nick Hynes says. “This could mean being able to infer how a food is prepared, such as stewed versus diced, or distinguish different variations of foods, like mushrooms or onions. The researchers are also interested in potentially developing the system into a ‘dinner aide’ that could determine what to cook given a dietary preference and a list of items in the fridge. “This could potentially help people figure out what’s in their food when they don’t have explicit nutritional information,” Hynes says. “For example, if you know what ingredients went into a dish but not the amount, you can take a photo, enter the ingredients, and run the model to find a similar recipe with known quantities, and then use that information to approximate your own meal.” PIC2RECIPE achieves a 65% success rate of picking out the right ingredients.
FT-aug17-eMag
To see the actual publication please follow the link above