Has mathematical reasoning in LLMs really advanced?
This study tests several SoTA models on a benchmark created with symbolic templates that enable diverse mathematical problems.
They find that LLMs exhibit variance when responding to variations of the same questions.
The performance of all the models declines by adjusting the numerical values in the question.
Another interesting finding is that as questions are made more challenging (e.g., increasing the number of clauses) the performance significantly deteriorates.
The authors hypothesize that the observed decline in performance is due to a lack of logical reasoning in current LLMs.
The study highlights the importance of model reliability and robustness and why it's important to continuously evaluate LLM systems after being deployed. This is not just for math-related problems, this also happened for use cases involving analysis, research, Q&A, and retrieval. We have seen that adjustments in prompts that have to deal with knowledge, numbers, retrieval, or structure can throw off the model so there is a need to trace LLMs and monitor them in production.
