Study finds artificial intelligence cannot predict decisions

Contemporary methods for predicting consumer choices have been found to be largely inaccurate, worsened by the finding that increasing methodological sophistication only leads to diminishing returns

In a 2013 report titled “The Future of Employment: How susceptible are jobs to computerisation?”, authors Carl Benedikt Frey and Michael A. Osborne examined how susceptible jobs are to computerisation. Using a Gaussian process classifier, they implemented a novel methodology to estimate the probability of computerisation for 702 detailed occupations.

The researchers concluded that 47pc of total US employment is at risk, which prompted another set of researchers to add information from the Bureau of Labor Statistics to further extract the probability of automation of all types of jobs. 

This effort has culminated in the ongoing platform known as “Will Robots Take My Job?”, which allows site visitors to learn how likely their occupations are to be replaced by robots and artificial intelligence (AI). For instance, the database concludes that there is 1.4pc chance that people trained in marketing – employed in demand generation, media planning, client servicing, media management, brand strategy roles to name a few – will be replaced by robots and AI.

The database further adds that a poll taken based on responses from website visitors suggests that there is a 38pc chance of marketers being replaced by robots and AI in the next two decades. Unfortunately for those contributing to the poll, the latest in academic research suggests that even this guess is far off by a century.

In a paper published with the Consumer Psychology Review, marketing professors David Gal and Itamar Simonson, respectively from the University of Illinois at Chicago and Stanford University, conclude that claims that AI can anticipate and predict consumer preferences with great accuracy are largely exaggerated. The report suggests that marketers spend less time in trying to  predict consumer choices with great accuracy and instead divert resources into the information environment that affects the choice of their products.

Examples of the aforementioned information environment predominantly feature user generated content in the form of user reviews, online recommendations, and new options, which the researchers argue have an increasing influence at the point of purchase decision and can neither be measured nor anticipated ex ante. 

They acknowledge that in some rare instances – such as the recommendation algorithms of Netflix or TikTok for instance – sophisticated methods and big data can improve predictions. The caveat is that these predictions are improved slightly and can be very imprecise, so much so that attempting to improve it is often a waste of effort. 

In the 2020 mass collaboration paper titled “Measuring the predictability of life outcomes with a scientific mass collaboration”, researchers tasked 160 teams to build predictive models around six life outcomes  using data from the Fragile Families and Child Wellbeing Study.

“Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model,” the authors concluded. “Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction.” 

Across a range of additional research, Gal and Simonson concluded that there is limited evidence that complex human behaviors can generally be predicted with much accuracy, adding that the evidence suggests that sophisticated methods do not dramatically increase the accuracy with which these behaviors can be predicted, which is the cornerstone of behavioral economics, which argues that framing and presentation has a sizable impact on driving purchase behavior.

“These findings should serve to limit our expectations of the degree to which consumer choices are predictable using current technologies,” said the researchers. “Like preferences, research suggests that many consumer attitudes are not stable, but are, to a degree, constructed at the time they are called for. As such, measures of attitudes (e.g., ratings) towards products or brands are likely to be noisy measures, and thus noisy predictors of future attitudes and preferences.”

The authors add that attitudes are particularly strong, stable, and well-defined, while noting that in the short-term, marketers and their activities are less likely to influence attitudes and so the focus should be on improving product consideration through proactive participation in the information environment.

This suggestion about focusing on the user generated content in the information environment is supported in “Understanding Behaviors that Lead to Purchasing” which analyzed the purchasing behavior of over two million Pinterest users to determine short-term and long-term signals in user behavior that suggest strong purchase intent.

“We found that users with long-term purchasing intent tend to save and click through more on content,” said the Pinterest study researchers, noting that user generated content drives consideration further down the purchase funnel. “However, as users approach the time of purchase their activity gets more topically focused and actions shift from saves to searches.”

These researchers found that purchase signals in online behavior can exist weeks before a purchase is made and is amplified in the last three days before a purchase.

Babar Khan Javed
Babar Khan Javed
Babar Khan Javed covers the advertising industry and marketing function in Pakistan for Profit. He can be reached on [email protected] with details about media, creative, and digital briefs, future projects, management changes, client wins or losses, and everything in between.

Must Read