Generative AI Confidently Wrong:
about elementary logic
Abstract: This note examines a July 2026 exchange with a paid version of ChatGPT concerning an elementary instance of disjunctive syllogism. ChatGPT made three distinct errors: it failed to recognize an explicitly stipulated pair of contradictories; presented ∃x(x=x) as an unproblematic formalization of “something exists”; and confused the truth of an argument’s premises with whether their truth had been independently demonstrated. When repeatedly asked whether the argument was sound, it moved from a qualified yes to no and then back to yes, without receiving any new relevant information.
The errors share a common pattern: fluent, formally presented claims unsupported by adequate justification and, in the relevant context, false. Corrections occurred only after direct pressure and challenge, not spontaneously. The exchange reveals persistent confusion about formalization, justification, validity, and soundness despite the elementary nature of the problem.
The case illustrates a central risk in relying on generative AI for logical analysis: correct and incorrect reasoning may be expressed with equal confidence, while successful correction depends on the reader already being able to detect the error. In my experience, errors like this remain pervasive across current versions of generative AI.
LLMs: The Illusion of Thinking