Brands are using emotional AI to enhance the customer's in-store experience

As AI assumes greater significance in our lives, its deployment in offline retail will only become bigger

Restaurant chain Pizza Hut recently deployed an artificial intelligence (AI)-powered mood detector that recommends dishes to customers based on their facial cues and expressions. The tool is “emotional AI” — another aspect of the ever widening reach of this breathtaking new technology.

In fact, this innovative way to woo customers at a quick service restaurant shows that while companies continue to use AI in an enterprise business environment, they are increasingly utilising it to enhance the consumer experience in a physical environment as well.

In Pizza Hut’s case, the way it works for a potential consumer is simple: You only need to stand in front of the device, look at the screen, and allow the detector to calibrate your mood before receiving pizza recommendations.

The device uses a statistical model that examines facial expressions by analysing eye movements, smiles, and frowns captured by the camera. These patterns are then compared with a database of publicly available images, ensuring accurate recommendations, explains Aanandita Datta, chief marketing officer, Pizza Hut India.

“Throughout the development of the AI mood detector, our primary focus has been on creating a seamless and safe experience. We prioritise the protection of our customers’ privacy, as the AI system stores face patterns as numerical data and does not retain images in any format. We have collaborated with Ebullient Gaming India to develop this AI tool,” Datta adds.

The tool allows a considerable degree of customisation to accommodate customers’ diverse preferences based on their mood, she explains.

Picking from a range of moods — happy, sad, angry, and others — the device provides recommendations from among 10 new pizzas started by the chain. Although these are early days, Pizza Hut says that it has received an enthusiastic response, especially from younger customers.

Meanwhile, in April, Reliance Retail introduced a “fragrance finder” at Tira, an omnichannel beauty store. The AI device helps consumers match fragrances closest to their preferences, allowing them to pick an olfactory note based on which a machine recommends a range of perfumes.

Similarly, outlets of Reliance’s fashion and lifestyle brand Azorte have tech interventions that include smart trial rooms. They are equipped with touch screen tablets that show the entire inventory available at the store. Once you click on the clothes of your choice, the staff brings them to the trial room.

The tablet allows one to select colour, price, and size. If you want to know how to pair clothes, the digital catalogue offers recommendations with the help of AI. The algorithm recognises the designer’s choice of pairing, and after being fed data collected through billing every week, it learns what customers want. Subsequently, any designer recommendation that is not in line with the purchasing pattern vanishes from suggestions for the customer concerned.

Rajat Mathur, a partner at BCG, notes that consumer-facing industries have made significant efforts in leveraging AI.

For example, banks have leveraged hyper-personalised nudges that are supported by a customer DNA of 2,000-plus behaviour variables. “These are curated, through AI and data, specifically for individual customers. That cannot happen with traditional analysis.”

Second, says Mathur, in retail, getting the price point right for a stock-keeping unit that serves a business purpose (profit or revenue maximisation) can be achieved through AI algorithms only. And third, leveraging from external data sources such as social media trends at a very granular level and supporting new product development can also help ensure that the seller is not out of stock.

Besides, image or computer vision and video analytics can provide real-time recommendations and help brands engage customers with a more targeted conversation.

The use of AI aids for customer engagement picked up during the Covid-19 pandemic as companies were forced to sell more products online, points out Ranjan Kumar, founder and chief executive officer of Entropik Technologies, a seven-year-old emotional AI company.

Historically, brands have understood consumers through two approaches — on the basis of surveys where respondents rate their products, and through direct interviews or focus group discussions, says Kumar. “The problem with both is that they are a stated response and are usually biased. So the brand doesn’t get to the core of what a consumer’s true perception or preference is.”

Entropik focuses on helping to understand the irrational, subconscious aspects of the consumer better. For the first three years, it built technology for facial coding, eye tracking and voice AI. “Face, voice and eyes are the three major sources through which we have learnt to understand consumers a lot better,” says Kumar.

Using its multimodal emotional AI technology, Entropik seeks to help its clients — among them P&G, Tata, Mondelez, Unilever, ITC, and Reckitt — understand their consumers better.

According to Kumar, examples such as Pizza Hut’s mood detector show brands experimenting with AI on live users — those walking into a physical store — to understand consumer perceptions.

In the case of apparel brands or a jewellery brand like Tanishq, trial mirrors are embedded with cameras that track facial expressions and give customers recommendations based on them, says Kumar. “These are some of the more live in-retail environment application areas of AI that have come up. While technology has matured, larger applications in the live scenarios are still limited, primarily because you are tracking consumers’ response using a camera, and you can do so only when it is consented to.”

Following the pandemic, Kumar observes, people are a lot more open to allowing camera access to gain a product experience. “It’s the same as people not having sales calls on Zoom earlier. Now, it’s perfectly fine to have business calls on Zoom with consent for it to be recorded,” he says, pointing out that a threshold has been crossed, and hence more brands will likely adopt the technology in the days ahead.

Different industries are at different phases of AI adoption. According to Kumar, sectors that are engaged in selling high-value assets accept emotional AI more readily. For instance, high-end automotive, jewellery and premium brands are more likely to embrace it than a retail hypermarket where the purchases are more functional than aspirational, emotional or impulsive.

As AI assumes greater significance in our lives and as companies look to match consumer preferences with in-store experiences, its deployment in offline retail will only become bigger.