If you've ever gone to the bar and had a song pop up on the playlist that you're dying to remember, then you've more than likely used Shazam, a smartphone app that "listens" to recorded music and identifies the track title and artist for you. Now, like something you'd see on an episode of Silicon Valley, MIT researchers have created a mobile app that can identify recipes based on food photos.
That's right, an app can now take your Insta-worthy eggs florentine from Sunday brunch and break it down into step-by-instructions for you to recreate at home. This seemingly impossible technology is called Pic2Recipe (hopefully this is a working title) and uses artificial intelligence to "look" at the photo and pinpoint a recipe for whatever dish is depicted.
MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) created a neural network called Recipe 1M that includes more than 1 million recipes and images. After training the AI with refined associations between the photos, recipes, and ingredients, it has frequently — although not entirely — been able to accurately pinpoint results; though the product is not meant for the market just yet.
The Verge gave the AI a test run and found many of its trial runs (potato chips, teriyaki beef ribs) drew no results. However, MIT remains steadfast in its mission: "You can imagine people using this to track their daily nutrition, or to photograph their meal at a restaurant and know what's needed to cook it at home later," Christoph Trattner, an assistant professor at MODUL University Vienna in the New Media Technology Department, told MIT News. "The team's approach works at a similar level to human judgment, which is remarkable."
As researchers continue to improve the capabilities of the app, their aim is to improve the system to a degree so that it can understand food in much more detail — from how it's prepared (diced, stewed, fried) to different variations (portabella vs cremini mushrooms).
If you want to give it a go, try the team's online demo. Or wait for the full-fledged app to drop, whenever that may be.