The outspoken vegan voiced his unease the Prime Minister is a fan of his former band during an interview with BBC Radio 4's Front Row programme.
Morrissey said it was a ''moral issue'' because Mr Cameron had hunted stag in the past.
He agreed with ex-bandmate Johnny Marr, who has previously forbade the Conservative leader from liking the band and called on him to stop saying he was a fan.
The singer said it was a ''mystery'' why the PM would like the Smiths.
He told presenter John Wilson he doubted whether he would bother opening the door if Mr Cameron turned up at his dressing room.
''I don't think I would. It's a moral issue. Killing a stag is like killing a child. What's the difference?''
The Prime Minister chose Smiths' track This Charming Man when he was a castaway on BBC Radio 4's Desert Island Discs four years ago.
Marr distanced himself from Mr Cameron four months ago with a message on Twitter which said: ''Stop saying that you like The Smiths, no you don't. I forbid you to like it.''
Asked whether was in tune with the guitarist, Morrissey said: ''I am indeed because David Cameron is a hunter and he kills stag, and so forth, and that's everything The Smiths were against.''
He conceded it was impossible to say who should listen to his music once it was released but added: ''Sometimes you see people whom you would least expect to be interested in it and they have a lifestyle which is everything that you oppose.
''And yet they're very influenced by something you've done. And you think 'well, how does it connect, where's the connection'?''
Morrissey has spoken of his dislike of the Tory Party throughout his career and his solo debut album in 1988 included a track called Margaret on the Guillotine - in which he mused about the execution of Baroness Thatcher.
He said: ''When you make music you're very happy that anybody listens, whoever they may be, but somebody with such an extreme lifestyle as David Cameron, killing stags as I say as a hobby, sport, you're mystified - you just can't see how it joins.''
The 51-year-old singer said the Prime Minister had been to see ''many, many Morrissey concerts'' but the pair had never met.
Morrissey also told how he was now refining his much-anticipated autobiography after completing his first draft, although he is still without a publisher.
''I've been through the whole life - in many ways - and I just wonder if 660 pages are too much for people to bear.
''So, I sit down and think are six pages too much for people to bear? I really don't know. I'm trimming at the moment.
''I'm really not that interesting, so I don't know why I've written so much.''