Food companies are using AI to nudge consumer behaviour

The authors set out their results against the following categories, which are based on the existing TIPPME framework (typology of interventions in proximal physical micro-environments):

• Placement
  • Availability (adding or removing products to the options offered)
    • For example, Danone tracks trends on social media and introduces new options in response, such as lactose-free options.
    • The Coca-Cola Company informs new product development using real-time sales data on flavour combinations that customers choose.
    • Seven & i Holdings uses AI to order products based on weather and other factors.
    • Nestlé uses digital imaging to monitor ice-cream freezers and help stores know when to order more.
    • PepsiCo uses store-level data based on demographics and consumer types to generate orders for stores.
  • Position (changing the order or proximity of options offered)
    • McDonald’s and the Coca-Cola Company both use digital menu boards that update based on real-time information such as weather, time of day, and sales data.
    • Yum! Brands (owner of KFC, Pizza Hut and more) uses facial recognition technology in some KFC restaurants in China to change the position of menu items based on a person’s estimated age, sex and apparent mood (read more here).
• Properties
  • Functionality (changing how products work or how people can interact with them)
    • PepsiCo and McDonald’s both report making additional product suggestions based on a customer’s current order: in the online store in the case of PepsiCo, and on drive-through menu boards in the case of McDonald’s. A note from TABLE: while we do not know the exact details of how these suggestions are produced, recommendation algorithms are already very commonly used across the internet, for example to suggest videos you might be interested in based on your viewing history, or to decide the order in which items appear in your social media feeds. See Wikipedia’s page on Collaborative filtering for more details.
    • Yum! Brands lets customers in India use the Alexa voice assistant, where they can “hear menu options, place an order, get promotional offers or even hear a chicken joke” (read more)
    • Nestlé uses a chatbot to deliver special offers.
    • PepsiCo and Yum! Brands are both using self-driving robots to deliver food items to customers, for example a “snackbot” on a university campus in California, a “robot waiter” in one Pizza Hut in Seoul, and delivery from some US Pizza Hut branches to a customer’s home via the FedEx SameDay Bot (read more here; it is unclear whether this planned trial has gone ahead or not, but a more recent story about a trial of autonomous delivery by Domino’s Pizza in Houston, Texas, here).
  • Presentation (changing the appearance or other properties of a product)
    • Several companies report using augmented reality or virtual reality in their marketing.
    • PepsiCo and Starbucks report tailoring adverts to consumers based on information such as device location (i.e. where the phone or computer you are using is). The information on PepsiCo comes from this source, which describes how consumer data from sources such as cookies, location and ID data (possibly a reference to mobile advertising identifiers) is used to target consumers in a more granular, individualised way. The information on Starbucks comes from its privacy policy, which also notes that Starbucks may gather information from third-party sources, for example information that users submit to “a blog, a chat room, or a social network like Facebook or Twitter”. 
    • Seven & i Holdings reports changing in-store advertising based on customers’ estimated age and gender, by using facial recognition.
  • Size (changing the size or shape of a product)
    • No examples were identified.
  • Information (adding or removing information about a product)
    • No examples were identified.

The authors note that previous interviews with people who have overseen “Big Data” applications in food contexts point to increasingly detailed uses for AI not identified in the current study, for example using high-resolution cameras to track eye movement and bodily cues and send relevant alerts to customers while they are in a store; or attaching cameras to frozen food aisle doors to change the promotions displayed based on eye tracking.

The authors raise concerns that AI-driven nudges could increase the selection of unhealthy foods. While they acknowledge that the study does not compare the effectiveness of AI versus non-AI driven “nudges”, they note that AI nudges have the potential to be tailored to individuals as opposed to just at the store level, for example. They also note that consumers are unlikely to be fully aware of the extent to which their personal data might be used in marketing, saying:

“As few consumers read or fully understand online terms and conditions, which are often lengthy and contain substantial ‘legalese’, the extent to which consumers actually consent to the collection and use of their data to inform dark nudges is highly questionable.”

Abstract

Objective:

To describe the use of artificial intelligence (AI)-enabled dark nudges by leading global food and beverage companies to influence consumer behaviour.

Design:

The five most recent annual reports (ranging from 2014–2018 or 2015–2019, depending on the company) and websites from 12 of the leading companies in the global food and beverage industry were reviewed to identify uses of AI and emerging technologies to influence consumer behaviour. Uses of AI and emerging technologies were categorised according to the Typology of Interventions in Proximal Physical Micro-Environments (TIPPME) framework, a tool for categorising and describing nudge-type behaviour change interventions (which has also previously been used to describe dark nudge-type approaches used by the alcohol industry).

Setting:

Not applicable.

Participants:

12 leading companies in the global food and beverage industry.

Results:

Text was extracted from 56 documents from 11 companies. AI-enabled dark nudges used by food and beverage companies included those that altered products and objects’ availability (e.g., social listening to inform product development), position (e.g., decision technology and facial recognition to manipulate the position of products on menu boards), functionality (e.g., decision technology to prompt further purchases based on current selections) and presentation (e.g., augmented or virtual reality to deliver engaging and immersive marketing).

Conclusions:

Public health practitioners and policymakers must understand and engage with these technologies and tactics if they are to counter industry promotion of products harmful to health, particularly as investment by the industry in AI and other emerging technologies suggests their use will continue to grow.

Reference

Brooks, R., Nguyen, D., Bhatti, A., Allender, S., Johnstone, M., Lim, C.P. and Backholer, K., Use of artificial intelligence to enable dark nudges by transnational food and beverage companies: analysis of company documents. Public Health Nutrition, pp.1-23.

Read the full paper here. See also the TABLE project Power in the food system: what’s powering the future of protein?