Categories: Apple

Apple’s study proves that LLM-based AI models are flawed because they cannot reason

A new paper from Apple’s artificial intelligence scientists has found that engines based on large language models, such as those from Meta and OpenAI, still lack basic reasoning skills.

Apple plans to introduce its own version of AI starting with iOS 18.1 – image credit Apple

The group has proposed a new benchmark, GSM-Symbolic, to help others measure the reasoning capabilities of various large language models (LLMs). Their initial testing reveals that slight changes in the wording of queries can result in significantly different answers, undermining the reliability of the models.

The group investigated the “fragility” of mathematical reasoning by adding contextual information to their queries that a human could understand, but which should not affect the fundamental mathematics of the solution. This resulted in varying answers, which shouldn’t happen.

Continue Reading on AppleInsider | Discuss on our Forums

Source: AppleInsider News

WBN

Share
Published by
WBN

Recent Posts

One Piece’s Anime Sets Sail Again in April

2025's going to be a year of One Piece, and kicks off with the anime…

15 hours ago

Elsbeth Season 2 Midseason Report Card: Murders, And Heists, And Bucket Hats, Oh My!

Do you think The Good Wife writers knew they had a fan-favorite character on their…

16 hours ago

James Bond’s Future Is Being Shaken Up by Corporate Clashes

Turns out, things aren't quite rosy for James Bond: the Broccolis and Amazon MGM can't…

16 hours ago

Your Keurig Coffee Pods Are Never Getting Recycled

A company is betting on aluminum to solve K-cups’ sustainability problem. Experts say it’s complicated.

17 hours ago

Ricoh Pentax in 2024: DSLR Woes, Compact Triumphs, and the Return of Film

What a fittingly unusual year for Ricoh Pentax, a photo company that itself is quite…

17 hours ago

BMW M8 Coupe to End Production in Early 2025: No MY26 Version Planned

A recent bulletin sent to BMW dealers confirms that production of the iconic BMW M8…

17 hours ago