Even the CIA knew it: "Trust, but Verify!". And in software, it is no different. One cannot expect software to be "Right First Time". Even ChatGPT will not always generate code that is perfect. And thus, I am a software test engineer, with a talent for intuitively coming across failures.
But, software is continuously getting better: This is because of the various feedback loops that exist in software now that we have automatic updates and people interested in all aspects of software development: some design, some code and debug, others write tests, and beta-testers shake out the errors and merely clearly describe them. This entire canvas of various skills and talents makes for a constant improvement in software.
And Tesla's Full Self-Driving software is even more entangled: the cars constantly feed back raw information about what the various sensors and cameras see, and aside from using it for the driving process itself, the Dojo computers at Tesla's backend train the network to better deal with any difficult situations. Believe it or not, but FSD11 is actually much better at driving than the average driver!