What does "not to have a prayer" mean?

It means you don't have a chance of success. In other words, even if you pray, you're still going to lose, fail, die, etc.