A mysterious novelty AI chatbot The so-called “gpt2 chatbot” caused a stir this week after it became available on a major large language model benchmarking site, LMSYS Org. No one knows where it came from, but many assume it has roughly the same capabilities as OpenAI’s GPT-4. This made gpt2-chatbot part of a rare class of AI models that only a handful of developers worldwide could achieve.
“Nobody knows who made it or what it is, but I’ve played with it a little and it seems to be at the same level of performance as GPT-4,” says Ethan Mollick, a professor of artificial intelligence at the University’s Wharton School of Pennsylvania, said in a tweet On Monday.
Just as the hype surrounding gpt2-chatbot was at its peak, the chatbot disappeared on Tuesday afternoon without much explanation. A note on the LMSYS site Home page is: “gpt2-chatbot is currently not available.” The short messages will take you there Policy pagewhich notes that model providers can withdraw anonymous AI models at any time, or as soon as an AI model receives enough votes.
Online AI communities love the anonymous gpt2 chatbot. An X user claims that gpt2-chatbot has programmed an almost perfect clone of the mobile game Flappy Bird. Another X user says it solved a problem Problem of the International Mathematical Olympiad In one shot. In long Reddit threads, users are wildly speculating about the origins of the gpt2 chatbot and debating whether it originated with him OpenAI, Googleor Anthropocene. There is no evidence to support these claims, but tweets from OpenAI CEO Sam Altman and other executives have only added fuel to the fire.
So far you have been able to try out the gpt2 chatbot yourself at Website of the LMSYS Org. It was available in the drop-down menu under “Direct Chat” or “Arena (side by side)”. LMSYS Org says in its Politics Blog that certain AI model developers can test anonymous, unpublished models before wider release. This has led many to believe that gpt2-chatbot is an anonymous model from a major AI developer.
“Just to be clear, as per our policy, we have worked with several model developers to bring their new models to our platform for community preview testing,” LMSYS Org said in a statement tweet on Monday, in response to a thread about gpt2-chatbot. “These models are for testing purposes only and will not be included in the leaderboard until they are released.”
LMSYS Org and OpenAI did not immediately respond to Gizmodo’s request for comment.
In Gizmodo’s limited testing, we found that the gpt2 chatbot has capabilities similar to leading AI models from Anthropic and OpenAI. It exhibited behavior unique to advanced large language models; it could reason well and create detailed plans for complicated tasks. Here are some of our examples comparing gpt2-chatbot (left) and Anthropic’s Claude Opus model (right).
A computer engineering professor at the University of Wisconsin found that gpt2-chatbot could perform a task that other leading AI models could not. Dimitris Papailiopoulos asked gpt2-chatbot to solve a math puzzle that involves learning some inexplicable rules. AI has great difficulty answering questions like these.
Ultimately, there is very little information available about the gpt2 chatbot. However, it seems clear that there is a powerful actor behind this AI model. The creator and origin of the gpt2 chatbot are expected to be announced in the coming weeks. This could mean that a new AI model is on the horizon, or perhaps a new AI developer is on the ground.