How tech giants are helping you find your favourite dress or shoes online

Firms like Amazon and eBay are using camera apps adept at recognising the pictures you've uploaded of the apparel you're dying to buy, and directing you to the relevant e-commerce site

In March this year, Amazon Inc, world largest e-commerce firm, announced a feature that enchanted the fashion community the world over. At the MARS 2019 Tech Conference in Pasadena, US, the company unveiled StyleSnap, an AI-powered tool that allows customers to discover clothes and shoes by uploading a picture or a screenshot of the same item from surroundings.

Headlines branded the tool as “Shazam for fashion”, after the popular app that lets users identify any sound track. 

It is simple and elegant. You don’t have to hunt for the label or brand of the hoodie your favourite celebrity is wearing on the cover of GQ Magazine. Just take a snap and upload it on the Amazon (or take a nap from within Amazon app), and it will find the same or similar looking article on Amazon.

Amazon isn’t the first tech company to use image recognition for clothes discovery and shopping. Global retailers like eBay, Wayfair, and Target have similar tools and technologies that got rolled out before Amazon’s StyleSnap. However, the accuracy of an image recognition engine depends on the amount of data (images, in this case) the software has seen and processed already, and undoubtedly Amazon is in an advantageous position when it comes to data.

In fact, search giant Google recently came out with massive upgrades in camera-based image recognition. The company added a ton of rich features to Google Lens, its image recognition tool, which was first rolled on Pixel phones in 2017.

Pull out your Android smartphone and tap Lens on the camera app, you’ll see three new icons. These represent Translate, for translating foreign language text; Dining, to discover restaurants in the vicinity by pointing camera on to a street, and Shopping. The shopping feature identifies clothes or shoes or accessories in front of the camera and redirects the user to the e-commerce sites where the said article(s) can be bought.

Though announced in 2018, these features went live only in June this year. Not only fashion apparel, Google is expanding the scope of Lens to many different things.

In July, it partnered with Indian e-commerce major Flipkart. Under the tie-up, Google Lens will recognise furniture items at Flipkart’s newly opened experience zones, serve them with information on the items and re-direct to Flipkart e-commerce site.

Pinterest, a social network built for images, also launched something called Pinterest Lens. The tool allows a user to search the thousands of posts and pictures on Pinterest through images, and offers shopping links for matched products wherever possible.

Global furniture retailer Ikea, which last year forayed into India, also has a similar AR app called Place app. Users can merely point their phone camera using the app at any piece of furniture and the app will automatically find similar Ikea items for sale. Snap the photograph of a chair or sofa, for example, and the app will automatically surface a list of Ikea pieces that match the visual description of the said items.

The above examples represent functional use-cases of modern image recognition technology. The basic science behind Google Lens to Amazon’s StyleSnap is the use of augmented reality (AR) to deliver more information on the subject that is placed in front of the camera.

Image recognition works on neural networks that process individual pixels of an image. Neural networks are layers of software code that work in a defined process. The system is built on complex codes that can accurately recognise an object, clicked from any angle or under any lighting condition.  

Though initially the system is fed with millions of images to learn from and recognise those, modern tools have built-in machine learning and AI algorithms that allow it to get better with more image searches.