Mr Hancock resigned last month after he was captured on camera kissing his close aide Gina Coladangelo in his ministerial office.
The motion was passed at a meeting of Newmarket Town Council, with five voting for, five against and four abstentions.
No Conservative councillor supported the motion.
Mayor of Newmarket Michael Jefferys, who is a member of the Labour Party, used his casting vote to pass the vote.
The passed resolution states that West Suffolk MP Mr Hancock has “neglected the best interests of his constituents” and, as health secretary, “demonstrated hypocrisy and hubris in the pursuit of his own interests”.
Mr Jefferys said he has been “dismayed” at Mr Hancock‘s performance as an MP and as health secretary, adding: “For someone in such a responsible position to then behave in the way that he has is for me the last straw.”
Mr Hancock was replaced by Sajid Javid who quit last year as Chancellor last year and is now one of the more experienced members of the Cabinet, having also worked at the Home Office.
West Suffolk Conservative Association said after Mr Hancock‘s resignation that it had received a “heartfelt apology” from him, and formally backed him to continue representing the constituency.
A spokesman for the association said in a statement on Wednesday: “Even the Labour/Independent-controlled local town council couldn’t find a majority against Matt Hancock.
“We are very grateful for the strong local support Matt is receiving.”
Additional reporting by PA