If a critical protocol bug was exploited and not found for a long time after being exploited (weeks or more) then it would cause a huge loss of confidence.
I still believe that the most likely cause of Bitcoin's death would be apathy. Everyone has to agree to stop supporting the project in order for it to die.