"It is always disappointing to lose hard working councillors", says Rishi Sunak