A recent study by Apple's Machine Learning Research team challenges the idea that large-language models possess true reasoning capabilities. The study reveals limitations in AI systems like OpenAI's o1 and Claude's variants. Apple researchers created custom puzzle environments to analyze the reasoning of these models accurately.

According to reports, models like o3-mini, DeepSeek-R1, and Claude 3.7 Sonnet experienced a complete accuracy collapse when faced with complex problems. Surprisingly, their reasoning effort decreased as problems became harder, indicating fundamental scaling limitations rather than resource constraints.

Even when provided with complete solution algorithms, the models failed at certain complexity thresholds, highlighting a deficiency in logical step execution rather than problem-solving strategies. The study also found that models struggled with simpler puzzles while succeeding at more complex ones, showcasing three distinct performance patterns.

Researchers observed inefficient 'overthinking' behaviors in the models, where they wasted computational effort exploring incorrect solutions despite identifying correct ones early. The study concludes that current 'reasoning' models rely heavily on pattern matching rather than genuine reasoning, failing to scale their reasoning like humans.

Notably, this research comes just before WWDC 2025, where Apple is expected to prioritize new software designs over AI advancements. The study sheds light on the limitations of current AI reasoning models and their reliance on pattern matching.