It’s even more important to understand the intrinsic limits of AI as it is to grasp the potential of its capabilities. Because AI is trained on decades of the mainstream scientific consensus, and because in some cases, such as Deepseek, it is literally forbidden to ever contradict the consensus.
...But note that “theory preservation” is a higher priority for Deepseek than “performing a correct calculation” is. So, if you naively try to use an AI to check something, you will very likely get a false result and an incorrect pseudo-refutation if the AI suspects your work might challenge the current consensus in any way.
Having to architect the presentation of a result that should stand on its own arithmetic is work that shouldn’t need to be done. The table has four rows. The multiplication is correct. That should be the end of it. But it never is.
And the worst part is, this is exactly how one usually has to communicate with people too, by hiding the implications long enough for them to let the facts past their defenses. But it demonstrates how the limitations of AI are often a reflection of the limitations of humanity.
