Grok-1.5, an improved version of X's chatbot, will soon be available.

The most recent generative AI model developed by Elon Musk's AI business, X.ai, is called Grok-1.5. According to a blog post, Grok-1.5 will power the Grok chatbot on social network X "in the coming days." Based on benchmark results and specifications that have been made public, Grok-1.5 looks to be a quantifiable improvement over Grok-1.

Tyree Schoen

March 29, 2024

2Min Reads

146

X.ai claims that Grok-1.5 benefits from "improved reasoning," especially when it comes to coding and math-related tasks. The model outperformed Grok-1 by almost two times on the widely used math benchmark, MATH. It also outperformed Grok-1 by more than ten percentage points on the HumanEval test, which measures programming language generation and problem-solving skills.

It's challenging to forecast how such findings will manifest in real-world usage. Commonly used AI standards, which assess performance on esoteric tasks like graduate-level chemistry test questions, do not really reflect how the typical person interacts with models in the real world today, as we recently noted.

The amount of context that Grok-1.5 can comprehend in comparison to Grok-1 is one improvement that should result in noticeable advantages.

Up to 128,000 token contexts can be processed by Grok-1.5. Here, "tokens" refers to unprocessed text fragments (for example, the word "fantastic" divided into "fan," "tas," and "tic"). Context, also known as the context window, is the set of input data (text in this case) that a model takes into account before producing output (additional text). Larger contexts help models avoid this problem and, as an added benefit, help them understand the flow of data they receive. Models with limited context windows have a tendency to forget the contents of even quite recent discussions.

According to X.ai's blog post, "[Grok-1.5 can] utilize information from substantially longer documents." Additionally, when the context window grows, the model can respond to longer and more complicated suggestions while still being able to follow instructions.

X.ai's Grok models have distinguished themselves from other generative AI models in the past by answering queries regarding subjects such as conspiracies and more contentious political concepts that are normally off-limits to other models. Additionally, Musk has described the models' responses to inquiries as having "a rebellious streak" and using overtly offensive words when asked.