Jail sentences of six months could be scrapped.
The prison minister has suggested that removing short jail terms could alleviate pressure on the system.
The move could mean that around 30,000 criminals per year in England and Wales could be spared jail under the proposals.
Short jail terms are frequently given to first time offenders or criminals convicted of offences such as petty thefts.
Exceptions would be made for violent or sex offenders, Rory Stewart told the Telegraph.
Stewart claimed that “very short” jail terms were “long enough to damage you and not long enough to heal you”.
He told the Daily Telegraph Magazine: “You bring somebody in for three or four weeks, they lose their house, their job, their family, their reputation.
“They come (into prison), they meet a lot of interesting characters (to put it politely) and then you whap them on to the streets again.
“The public are safer if we have a good community sentence… and it will relieve a lot of pressure on prisons.”
Peter Dawson, director of the Prison Reform Trust, praised ministers, adding they “should be congratulated for having the political courage to start the debate”.
The prison population has doubled since the early 1990s, soaring from around 40,000 to more than 80,000 in 2018.
More than half of the 86,275 offenders sentenced to immediate custody in England and Wales in 2017 were sentenced to six months or less, according to a Parliamentary response from Mr Stewart to shadow justice secretary Richard Burgon last month.
Mr Stewart claimed research indicated that criminals given short jail terms were more likely to re-offend than those given community service.
A Ministry of Justice spokeswoman said: “As we have said previously, short sentences are too often ineffective, provide little opportunity to rehabilitate offenders and lead to unacceptably high rates of re-offending.
“That’s why we are exploring potential alternatives but this work is ongoing and we have reached no conclusions at this time.”
In Scotland, where judges are guided by a legal presumption against sentences of less than six months, re-offending rates are said to have fallen to their lowest levels for nearly two decades.
According to Humza Yousaf, the SNP’s Justice Secretary, a shift to greater use of community sentences has contributed to a 19-year low in the re-conviction rate north of the border.
The Scottish Government is due to extend the presumption against short prison sentences to 12 months in a drive to further reduce re-offending and keep crime down.