NewtonX research finds reliability is the determining factor when buying AI, but is brand awareness coloring perceptions?

Get the download from Winston Ford, NewtonX Senior Product Manager:

“Rank these eight key decision-making criteria, with 10 being the most important and 1 being the least important.“

When we asked the 240 executives who took part in our NewtonX Prime Generative AI survey to do this, one criterion stood head and shoulders above the rest. 52.1% ranked reliability as their chosen generative AI tool’s most important attribute. 

In comparison, functionality and value only got 39.2% and 36.7% of the vote respectively. And, pricing, scalability, and the vendor’s reputation were at the bottom of the list, ranked most important by only 23.8%, 22.1%, and 20.8% of respondents respectively. 

Given the risks should AI malfunction —  Zillow, for instance, lost $8.3 billion because its algorithm was overvaluing properties — it’s not surprising that decision-makers would rate reliability so highly. 

But the results beg two important questions. How do you objectively evaluate a generative AI tool’s reliability? And how do you determine whether it meets an acceptable standard?

Data scientists typically use mathematical formulas such as F1 score to measure AI models’ performance. But our research suggests the less objective metric of brand awareness may also be having an impact in the boardroom. 

ChatGPT, currently used by 81.3% of our survey participants, was ranked 7 or above for reliability by an overwhelming majority: 80.4% of respondents.

By contrast, the lower the awareness of a generative AI tool was among our survey participants, the less likely they were to rate it highly for reliability. Generative AI tools with low awareness also have negative net promoter scores.

Of course, this might change as awareness increases. But that begs two other questions. Will newer generative AI tools ever be able to challenge ChatGPT’s ubiquity? And what will it take to do so?