OpenAI did not invent AI chatbots, but ChatGPT is the product that made them mainstream. ChatGPT 3.5’s launch in November 2022 is widely regarded as the moment generative AI became widely accessible, quickly gaining over 100 million monthly active users. It introduced general-purpose large language models capable of human-like dialogue to everyday users, sparking a rapid expansion of the AI chatbot market, with competitors like Google’s Bard (now renamed Gemini), Anthropic’s Claude, and Meta’s LLaMA emerging shortly after.
ChatGPT was the first AI chatbot many people interacted with directly, becoming synonymous with AI itself and dominating the market with around 800 million weekly active users, accounting for nearly half of all AI chatbot usage worldwide. However, in a recent human-centered evaluation called the Humaine study by British company Prolific, ChatGPT ranked only eighth, beaten by Google’s Gemini models, DeepSeek, xAI’s Grok, and the French AI bot from Mistral AI. Prolific argues that traditional benchmarks focus too much on technical performance and fail to measure what users truly value in chatbot interactions.
The Humaine benchmark evaluates several key human-centric factors: understanding user input, maintaining conversational context, clear and engaging communication style, and trustworthiness including ethics and safety considerations. The study uses a robust participant pool who engaged in multi-turn conversations with anonymized chatbots, assessing them in head-to-head comparisons. Participants’ feedback formed the leaderboard scores, emphasizing real-world user experience over narrow technical tests.
Despite ChatGPT’s market dominance, its eighth-place finish highlights the difference between popularity and perceived user satisfaction on certain qualitative metrics. Google’s Gemini 2.5 Pro topped the Humaine leaderboard, earning high marks across the board and reinforcing its position as a leading AI model. DeepSeek, a Chinese-developed chatbot, secured second place, demonstrating strong performance especially in communication style, while Mistral AI’s French chatbot claimed third, noted for its adaptiveness and engaging conversation flow. xAI’s Grok, despite earlier controversies, ranked fourth and fifth and showed progress in trust and ethics, benefiting from updates that reduced problematic responses.
Other notable AI models fared less well; Anthropic’s Claude, often praised in other assessments, placed 11th, while Meta’s LLaMA models ranked in the lower half despite high expectations. Lesser-known bots from Chinese and Canadian startups scored lower as well. The study is ongoing and continues to evolve as it incorporates more participant data and newer AI models. It brings a unique emphasis on the human-facing qualities of AI interactions—how well chatbots connect with and support users—in contrast to purely technical or skill-based evaluations. OpenAI did receive recognition for ChatGPT in one of Humaine’s award categories, highlighting its proactive initiative in conversations, even if it did not top overall rankings. This reflects the complexity of assessing AI, where multiple dimensions of user experience matter beyond raw capability or popularity.
