In 2004, both the Vajpayee government at the center, and the Naidu government in Andhra Pradesh were voted out by the people of India.
This happened in spite of the fact that both government performed well during their five-year term.
Why did this happen? See this episode and you will see that the Indian public remain unforgiving, if pushed to the brink. Then political loyalties are thrown to the wind.
You could extend the same to todays Uttar Pradesh where Yogi Aditya Nath is facing similar challenges.
See the full episode for details.