Elon Musk’s AI chatbot can now “understand” images, including charts and graphs full of information. Sorry, doesn’t everyone use the platform once known as Twitter for multidisciplinary research and streamlining their workflows?
Introduced as Grok-1.5V – or Grok 1.5 “Vision”, the company’s “first-generation multimodal model” – the bot can not only respond to your uploaded images and screenshots, but also reason through complex documents, scientific charts and graphs. Screenshots and photos, the company says. Additionally, Grok-1.5V will gain “realistic spatial understanding” to better understand the physical world represented in the images uploaded by its users.
“Advancing our multimodal understanding and generation capabilities are important steps in building a useful AGI that can understand the universe,” the company wrote in its announcement. “In the coming months, we expect to significantly improve both capabilities, across multiple modalities such as images, audio and video.”
According to the report, if you are a paying X user, Elon Musk wants his Grok AI to write your posts for you
Example use cases include translating a diagram into Python code, turning a child’s drawing into a bedroom story, locating the largest object in a group of many objects, and telling a driver whether they have enough room to avoid an obstacle .
Grok-1.5V is released alongside xAI’s RealWorldQA, an image and input dataset designed to test other GenAI models against Grok’s real-world reasoning.
The tweet may have been deleted
However, competition is the least of Grok’s worries. Despite continued investment in xAI, Grok has yet to reach its first users and employees – a new report claims that its own developers are struggling to use the slow xAI API. The same report published by Assets This week highlighted concerns from X employees that Musk suggested Grok write paid user posts for them, despite warnings from developers and employees. Last week, Grok came under fire for generating fake news headlines from an alternate reality in which Iran had attacked Tel Aviv with a military arsenal – not for the first time.
While GenAI chatbots hallucinating realities and generating fake news are par for the course, Grok’s faux pas points to another site-wide problem. The bot, a typical Musk response to ChatGPT, integrates into a platform whose defenses against faulty AI have slowly weakened. Combined with the overall bad reputation of
Grok-1.5V will be available soon to early testers and select users.